Systems and methods of registration for image-guided surgery

ABSTRACT

A system includes a catheter, a display system, and a control system in communication with the catheter and the display system. The control system includes one or more processors configured for receiving a set of model points of a model of one or more passageways of a patient and receiving a first set of measures points collected from within the patient passageways. Each point includes coordinates within a surgical environment occupied by the patient. The one or more processors are also configured for generating a first registration between the set of measured points and the set of model points, generating a second registration between a second set of measured points and the set of model points, and determining whether to implement the second registration in place of the first registration.

RELATED APPLICATIONS

This patent application claims priority to and the benefit of the filingdate of U.S. Provisional Patent Application 62/205,440, entitled“SYSTEMS AND METHODS OF REGISTRATION FOR IMAGE-GUIDED SURGERY,” filedAug. 14, 2015, which is incorporated by reference herein in itsentirety.

FIELD

The present disclosure is directed to systems and methods for conductingan image-guided procedure, and more particularly to systems and methodsfor displaying pathology data for tissue sampled during an image-guidedprocedure.

BACKGROUND

Minimally invasive medical techniques are intended to reduce the amountof tissue that is damaged during medical procedures, thereby reducingpatient recovery time, discomfort, and deleterious side effects. Suchminimally invasive techniques may be performed through natural orificesin a patient anatomy or through one or more surgical incisions. Throughthese natural orifices or incisions clinicians may insert minimallyinvasive medical instruments (including surgical, diagnostic,therapeutic, or biopsy instruments) to reach a target tissue location.To assist with reaching the target tissue location, the location andmovement of the medical instruments may be correlated with pre-operativeor intra-operative images of the patient anatomy. With the image-guidedinstruments correlated to the images, the instruments may navigatenatural or surgically created passageways in anatomic systems such asthe lungs, the colon, the intestines, the kidneys, the heart, thecirculatory system, or the like. Traditional instrument tracking andreferencing systems may require the use of patient pads duringpre-operative and operative imaging and may disturb the clinicalenvironment or workflow. Systems and methods for performingimage--guided surgery with minimal clinical disturbances are needed.

SUMMARY

The embodiments of the invention are best summarized by the claims thatfollow the description.

However, an exemplary method may include segmenting a set of firstmodality image data representing a model of one or more passagewayswithin a patient and generating a first set of points based on thesegmented set of first modality image data representing the model of theone or more passageways. The method may further include determining aset of matches between a second set of points and the first set ofpoints, wherein the second set of points is obtained by a secondmodality and discarding a subset of the set of matches based on a firstheuristic to generate a modified set of matches.

Another exemplary method may include receiving a set of model points ofa model of one or more passageways of a patient and receiving a set ofmeasured points collected from within the patient passageways, eachpoint comprising coordinates within a surgical environment occupied bythe patient. Weights may be assigned to one or more of the measuredpoints. The method may further include matching each measured point to amodel point to generate a set of matches, a value of each of the matchesdepending on the assigned weight of the measured point in the match, andmoving the set of measured points relative to the set of model pointsbased on the set of matches.

Another exemplary method may include receiving a set of measured pointscollected from within the patient passageways, each point comprisingcoordinates within a surgical environment occupied by the patient, andidentifying features of the patient passageways based on the set ofmeasured points. The method may further include steps or operations ofidentifying corresponding features, to the identified features, in amodel of the patient passageways obtained prior to receiving the set ofmeasured points, and of performing an initial registration of the set ofmeasured points to a set of modeled points obtained from the model.

An addition exemplary method may include accessing a set of model pointsof a model of one or more passageway's of a patient, detecting a pointcollection condition in data obtained from a catheter, initiatingcollection of a set of measured points, and performing a point setregistration algorithm using the set of model points and the set ofmeasured points.

Another additional exemplary method may include receiving a set of modelpoints of a model of one or more. passageway's of a patient andreceiving a first set of measured points collected from within thepatient passageways, each point including coordinates within a surgicalenvironment occupied by the patient. The method may further includeoperations of generating a first registration between the set ofmeasured points and the set of model points, generating a secondregistration between a second set of measured points and the set ofmodel points, and then determining whether to implement the secondregistration in place of the first registration.

Another exemplary method may include receiving a set of model points ofa model of one or more passageways of a patient and determining a stateof a catheter positioned within the one or more passageways of thepatient. When the state of the catheter satisfies a condition, themethod. may further include collecting a set of measured. points fromwithin the patient passageways, each point comprising coordinates withina surgical environment occupied by the patient, and then generating aregistration between the set of measured points and the set of modelpoints.

Yet another exemplary method may include receiving a set of model pointsof a model of one or more passageways of a patient and receiving a firstset of measured points collected from within the patient passageways,each point comprising coordinates within a surgical environment occupiedby the patient. The method may further include generating a firstregistration between the set of measured points and the set of modelpoints, detecting a motion of the patient, and generating a secondregistration between a second set of measured points and the set ofmodel points.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory innature and are intended to provide an understanding of the presentdisclosure without limiting the scope of the present disclosure. In thatregard, additional aspects, features, and advantages of the presentdisclosure will be apparent to one skilled in the art front thefollowing detailed description.

BRIEF DESCRIPTIONS OF THE DRAWINGS

Aspects of the present disclosure are best understood from the followingdetailed description when read with the accompanying figures. It isemphasized that, in accordance with the standard practice in theindustry, various features are not drawn to scale. In fact, thedimensions of the various features may he arbitrarily increased orreduced for clarity of discussion. In addition, the present disclosuremay repeat reference numerals and/or letters in the various examples.This repetition is for the purpose of simplicity and clarity and doesnot in itself dictate a relationship between the various embodimentsand/or configurations discussed,

FIG. 1 is a teleoperated medical system, in accordance with embodimentsof the present disclosure.

FIG. 2A illustrates a medical instrument system utilizing aspects of thepresent disclosure.

FIG. 2B illustrates a distal end of the medical instrument system ofFIG. 2A with an extended medical tool, in accordance with embodiments ofthe present disclosure.

FIG. 3 illustrates the distal end of the medical instrument system ofFIG. 2A positioned within a human lung.

FIG. 4 is a flowchart illustrating a method used to provide guidance inan image-guided surgical procedure according to an embodiment of thepresent disclosure.

FIGS. 5A, 5B, and 5C illustrate steps in segmentation process thatgenerates a model of a human lung for registration according to anembodiment of the present disclosure.

FIGS. 6A and 6B are side views of a patient coordinate space including amedical instrument mounted on an insertion assembly according to anembodiment of the present disclosure.

FIG. 6C is a side views of a patient in a patient coordinate spaceincluding an endotracheal tube according to an embodiment of the presentdisclosure.

FIG. 6D includes diagrams of an interior surface of the endotrachealtube of FIG. according to aspects of the present disclosure.

FIG. 7 illustrates a flowchart of a portion of an image-guided surgicalprocedure according to an embodiment of the present disclosure.

FIG. 8 illustrates two sets of points representing anatomy that are tobe registered as part of an image-guided surgical according to anembodiment of the present disclosure.

FIG. 9 illustrates a seeding process of an image-guided surgicalprocedure according to an embodiment of the present disclosure.

FIG. 10 illustrates a registration of the two sets of points resultingfrom a registration technique according to an embodiment of the presentdisclosure

FIGS. 11A, 11B, 11C, 11D, 11E, 11F, 11G, and 11H illustrate variousapproaches for registering two sets of points representing patientanatomy according to embodiments of the present disclosure.

FIG. 12 illustrates a display stage of a registration techniqueaccording to an embodiment of the present disclosure.

FIG. 13 illustrates a point pool stored in memory according to anembodiment of the present disclosure.

FIG. 14 illustrates a flowchart of a portion of an image-guided surgicalprocedure according to an embodiment of the present disclosure.

DETAILED DESCRIPTION

In the following detailed description of the aspects of the invention,numerous specific details are set forth in order to provide a thoroughunderstanding of the disclosed embodiments. However, it will be obviousto one skilled in the art that the embodiments of this disclosure may bepracticed without these specific details. In other instances well knownmethods, procedures, components, and circuits have not been described indetail so as not to unnecessarily obscure aspects of the embodiments ofthe invention. And, to avoid needless descriptive repetition, one ormore components or actions described in accordance with one illustrativeembodiment can he used or omitted as applicable from other illustrativeembodiments.

The embodiments below will describe various instruments and portions ofinstruments in terms of their state in three-dimensional space. As usedherein, the term “position” refers to the location of an object or aportion of an object in a three-dimensional space (e.g., three degreesof translational freedom along Cartesian x-, y-, and z-coordinates). Asused herein, the term “orientation” refers to the rotational placementof an object or a portion of an object (three degrees of rotationalfreedom—e.g., roll, pitch, and yaw). As used herein, the term “pose”refers to the position of an object or a portion of an object in atleast one degree of translational freedom and to the orientation of thatobject or portion of the object in at least one degree of rotationalfreedom (up to six total degrees of freedom). As used herein, the term“shape” refers to a set of poses, positions, or orientations measuredalong an object.

Referring to FIG. 1 of the drawings, a teleoperated medical system foruse in, for example, surgical, diagnostic, therapeutic, or biopsyprocedures, is generally indicated by the reference numeral 100. Asshown in FIG. 1, the teleoperated system 100 generally includes ateleoperational manipulator assembly 102 for operating a medicalinstrument 104 in performing various procedures on the patient P. Theassembly 102 is mounted to or near an operating table O. A masterassembly 106 allows the clinician or surgeon S to view theinterventional site and to control the slave manipulator assembly 102.

The master assembly 106 may be located at a surgeon's console which isusually located in the same room as operating table 0. However, itshould be understood that the surgeon S can be located in a differentroom or a completely different building from the patient P. Masterassembly 106 generally includes one or more control devices forcontrolling the manipulator assemblies 102. The control devices mayinclude any number of a variety of input devices, such as joysticks,trackballs, data gloves, trigger-guns, hand-operated controllers, voicerecognition devices, body motion or presence sensors, or the like. Insome embodiments, the control devices will be provided with the samedegrees of freedom as the associated medical instruments 104 to providethe surgeon with telepresence, or the perception that the controldevices are integral with the instruments 104 so that the surgeon has astrong sense of directly controlling instruments 104. In otherembodiments, the control devices may have more or fewer degrees offreedom than the associated medical instruments 104 and still providethe surgeon with telepresence. In some embodiments, the control devicesare manual input devices which move with six degrees of freedom, andwhich may also include an actuatable handle for actuating instruments(for example, for closing grasping jaws, applying an electricalpotential to an electrode, delivering a medicinal treatment, or thelike).

The teleoperational assembly 102 supports the medical instrument system104 and may include a kinematic structure of one or more non-servocontrolled links (e.g., one or more links that may be manuallypositioned and locked in place, generally referred to as a set-upstructure) and a teleoperational manipulator. The teleoperationalassembly 102 includes plurality of actuators or motors that drive inputson the medical instrument system 104 in response to commands from thecontrol system (e.g., a control system 112). The motors include drivesystems that when coupled to the medical instrument system 104 mayadvance the medical instrument into a naturally or surgically createdanatomic orifice. Other motorized drive systems may move the distal endof the medical instrument in multiple degrees of freedom, which mayinclude three degrees of linear motion e.g., linear motion along the X,Y, Z Cartesian axes) and in three degrees of rotational motion (e.g.,rotation about the X, Y, Z Cartesian axes). Additionally, the motors canbe used to actuate an articulable end effector of the instrument forgrasping tissue in the jaws of a biopsy device or the like. Motorposition sensors such as resolvers, encoders, potentiometers, and othermechanisms may provide sensor data to the teleoperational assemblydescribing the rotation and orientation of the motor shafts. Thisposition sensor data may be used to determine motion of the objectsmanipulated by the motors.

The teleoperational medical system 100 also includes a sensor system 108with one or more sub--systems for receiving information about theinstruments of the teleoperational assembly. Such sub-systems mayinclude a position/location sensor system (e.g., an electromagnetic (EM)sensor system); a shape sensor system for determining the position,orientation, speed, velocity, pose, and/or shape of the catheter tipand/or of one or more segments along a flexible body of instrumentsystem 104; and/or a visualization system for capturing images from thedistal end of the catheter system.

The visualization system (e.g., visualization system 231 of FIG. 2A) mayinclude a viewing scope assembly that records a concurrent or real-timeimage of the surgical site and provides the image to the clinician orsurgeon S. The concurrent image may be, for example, a two or threedimensional image captured by an endoscope positioned within thesurgical site. In this embodiment, the visualization system includesendoscopic components that may he integrally or removably coupled to themedical instrument 104. However in alternative embodiments, a separateendoscope, attached to a separate manipulator assembly may be used withthe medical instrument to image the surgical site. The visualizationsystem may be implemented as hardware, firmware, software or acombination thereof which interact with or are otherwise executed by oneor more computer processors, which may include the processors of acontrol system 112 (described below). The processors of the controlsystem 112 may execute instructions comprising instruction correspondingto processes disclosed herein.

The teleoperational medical system 100 also includes a display system110 f©r displaying an image or representation of the surgical site andmedical instrument system(s) 104 generated by sub-systems of the sensorsystem 108. The display 110 and the operator input system 106 may beoriented so the operator can control the medical instrument system 104and the operator input system 106 with the perception of telepresence.

The display system 110 may also display an image of the surgical siteand medical instruments captured by the visualization system. Thedisplay 110 and the control devices may be oriented such that therelative positions of the imaging device in the scope assembly and themedical instruments are similar to the relative positions of thesurgeon's eyes and hands so the operator can manipulate the medicalinstrument 104 and the hand control as if viewing the workspace insubstantially true presence. By true presence, it is meant that thepresentation of an image is a true perspective image simulating theviewpoint of an operator that is physically manipulating the instrument104.

Alternatively or additionally, the display 110 may present images of thesurgical site recorded pre-operatively or intra-operatively using imagedata from imaging technology such as, computed tomography (CT), magneticresonance imaging (MRI), fluoroscopy, thermography, ultrasound, opticalcoherence tomography (OCT), thermal imaging, impedance imaging, laserimaging, or nanotube X-ray imaging. The pre-operative or intra-operativeimage data may be presented as two-dimensional, three-dimensional, orfour-dimensional (including e.g., time based or velocity basedinformation) images or as images from models created from thepre-operative or intra-operative image data sets.

In some embodiments often for purposes of imaged guided surgicalprocedures, the display 110 may display a virtual navigational image inwhich the actual location of the medical instrument 104 is registered(i.e., dynamically referenced) with the preoperative or concurrent.images/model to present the clinician or surgeon S with a virtual imageof the internal surgical site from the viewpoint of the location of thetip of the instrument 104. An image of the tip of the instrument 1.04 orother graphical or alphanumeric indicators may be superimposed on thevirtual image to assist the surgeon controlling the medical instrument.Alternatively, the instrument 104 may not be visible in the virtualimage.

In other embodiments, the display 110 may display a virtual navigationalimage in which the actual location of the medical instrument isregistered with preoperative or concurrent images to present theclinician or surgeon S with a virtual image of medical instrument withinthe surgical site from an external viewpoint. An image of a. portion ofthe medical instrument or other graphical or alphanumeric indicators maybe superimposed on the virtual image to assist the surgeon controllingthe instrument 104. As described herein, visual representations of datapoints may be rendered to the display 110. For example, measured datapoints, moved data points, registered data points, and other data pointsdescribed herein may be displayed on the display 110 in a visualrepresentation. The data points may be visually represented in a userinterface by a plurality of points or dots on the display or as arendered model, such as a mesh or wire model created based on the set ofdata points. In some embodiments, a visual representation may berefreshed in the display 110 after each processing operations has beenimplemented to alter the data points.

The teleoperational medical system 100 also includes a. control system112. The control system 112 includes at least one memory and at leastone computer processor (not shown), and typically a plurality ofprocessors, for effecting control between the medical instrument system104, the operator input system 106, the sensor system. 108, and thedisplay system 110. The control system 112 also includes programmedinstructions (e.g., a computer-readable medium storing the instructions)to implement some or all of the methods described in accordance withaspects disclosed herein, including instructions for providingpathological information to the display system 110. While control system112 is shown as a single block in the simplified schematic of FIG. 1,the system may include two or more data processing circuits with oneportion of the processing optionally being performed on or adjacent theteleoperational assembly 102, another portion of the processing beingperformed at the operator input system 106, and the like. Any of a widevariety of centralized or distributed data processing architectures maybe employed. Similarly, the programmed instructions may be implementedas a number of separate programs or subroutines, or they may beintegrated into a number of other aspects of the teleoperational systemsdescribed herein. In one embodiment, control system 112 supportswireless communication protocols such as Bluetooth, IrDA, HomeRF, IEEE802.11, DECT, and Wireless Telemetry,

In some embodiments, control system 112 may include one or more servocontrollers that receive force and/or torque feedback from the medicalinstrument system 104. Responsive to the feedback, the servo controllerstransmit signals to the operator input system 106. The servocontroller(s) may also transmit signals instructing teleoperationalassembly 102 to move the medical instrument system(s) 104 which extendinto an internal surgical site within the patient body via openings inthe body. Any suitable conventional or specialized servo controller maybe used. A servo controller may be separate from, or integrated with,teleoperational assembly 102. In some embodiments, the servo controllerand teleoperational assembly are provided as part of a teleoperationalarm cart positioned adjacent to the patient's body.

The control system 112 may further include a virtual visualizationsystem to provide navigation assistance to the medical instrumentsystem(s) 104 when used in an image-guided surgical procedure. Virtualnavigation using the virtual visualization system is based uponreference to the acquired preoperative or intraoperative dataset of theanatomic passageways. More specifically, the virtual visualizationsystem processes images of the surgical site imaged using imagingtechnology such as computerized tomography (CT), magnetic resonanceimaging (MRI), fluoroscopy, thermography, ultrasound, optical coherencetomography (OCT), thermal imaging, impedance imaging, laser imaging,nanotube X-ray imaging, or the like. Software alone or in combination.with manual input is used to convert the recorded images into segmentedtwo dimensional or three dimensional composite representation of apartial or an entire anatomic organ or anatomic region. An image dataset is associated with the composite representation. The compositerepresentation and the image data set describe the various locations andshapes of the passageways and their connectivity. The images used togenerate the composite representation may be recorded preoperatively orintra-operatively during a clinical procedure. In an alternativeembodiment, a virtual visualization system may use standard.representations (i.e., not patient specific) or hybrids of a standardrepresentation and patient specific data. The composite representationand any virtual images generated by the composite representation mayrepresent the static posture: of a deformable anatomic region during oneor more phases of motion (e.g., during an inspiration/expiration cycleof a lung).

During a virtual navigation procedure, the sensor system 108 may be usedto compute an approximate location of the instrument with respect to thepatient anatomy. The location can be used to produce both macro-level(external) tracking images of the patient anatomy and virtual internalimages of the patient anatomy. Various systems for using electromagnetic(EM) sensor, fiber optic sensors, or other sensors to register anddisplay a medical implement together with preoperatively recordedsurgical images, such as those from a virtual visualization system, areknown. For example U.S. patent application Ser. No. 13/107,562 (filedMay 13, 2011) (disclosing “Medical System Providing Dynamic Registrationof a Model of an Anatomic Structure for image-Guided Surgery”) which isincorporated by reference herein in its entirety, discloses one suchsystem.

The teleoperational medical system 100 may further include optionaloperation and support systems (not shown) such as illumination systems,steering control systems, irrigation systems, and/or suction systems. Inalternative embodiments, the teleoperational system may include morethan one teleoperational assembly and/or more than one operator inputsystem. The exact number of manipulator assemblies will depend on thesurgical procedure and the space constraints within the operating room,among other factors. The operator input systems may be collocated orthey may be positioned in separate locations. Multiple operator inputsystems allow more than one operator to control one or more manipulatorassemblies in various combinations.

FIG. 2A illustrates a medical instrument system 200, which may be usedas the medical instrument system 104 in an image-guided medicalprocedure performed with teleoperational medical system 100.Alternatively, the medical instrument system 200 may be used fornon-teleoperational exploratory procedures or in procedures involvingtraditional manually operated medical instruments, such as endoscopy.Additionally or alternatively the medical instrument system 200 may beused to gather (i.e., measure) a set of data points corresponding tolocations with patient anatomic passageways.

The instrument system 200 includes a catheter system 202 coupled to aninstrument body 204. The catheter system 202 includes an elongatedflexible catheter body 216 having a proximal end 217 and a distal end ortip portion 218. In one embodiment, the flexible body 216 has anapproximately 3 mm outer diameter. Other flexible body outer diametersmay be larger or smaller. The catheter system 202 may optionally includea shape sensor 222 for determining the position, orientation, speed,velocity, pose, and/or shape of the catheter tip at distal end 218and/or of one or more segments 224 along the body 216. The entire lengthof the body 216, between the distal end 218 and the proximal end 217,may be effectively divided into the segments 224. If the instrumentsystem 200 is a medical instrument system 104 of a teleoperationalmedical system 100, the shape sensor 222 may be a component of thesensor system 108. If the instrument system 200 is manually operated orotherwise used for non-teleoperational procedures, the shape sensor 222may be coupled to a tracking system 230 that interrogates the shapesensor and processes the received shape data.

The shape sensor 222 may include an optical fiber aligned with theflexible catheter body 216 (e.g., provided within an interior channel(not shown) or mounted externally). In one embodiment, the optical fiberhas a diameter of approximately 200 um. In other embodiments, thedimensions may be larger or smaller. The optical fiber of the shapesensor system 222 forms a fiber optic bend sensor for determining theshape of the catheter system 202. In one alternative, optical fibersincluding Fiber Bragg Gratings (FBGs) are used to provide strainmeasurements in structures in one or more dimensions. Various systemsand methods for monitoring the shape and relative position of an opticalfiber in three dimensions are described in U.S. patent application Ser.No. 11/180,389 (tiled Jul. 13, 2005) (disclosing “Fiber optic positionand shape sensing device and method relating thereto”); U.S. patentapplication Ser. No. 12/047,056 (filed on Jul. 16, 2004) (disclosing“Fiber-optic shape and relative position sensing”); and U.S. Pat. No.6,389,187 (filed on Jun. 17, 1998) (disclosing “Optical Fibre BendSensor”), which are all incorporated by reference herein in theirentireties. Sensors in alternative embodiments may employ other suitablestrain sensing techniques, such as Rayleigh scattering, Ramanscattering, Brillouin scattering, and Fluorescence scattering. In otheralternative embodiments, the shape of the catheter may he determinedusing other techniques. For example, the history of the catheter'sdistal tip pose can be used to reconstruct the shape of the device overthe interval of time. As another example, historical pose. position. ororientation data may be stored for a known point of an instrument systemalong a cycle of alternating motion, such as breathing. This stored datamay he used to develop shape information about the catheter.Alternatively, a series of positional sensors, such as electromagnetic(EM) sensors, positioned along the catheter can be used for shapesensing. Alternatively, a history of data from a positional sensor, suchas an EM sensor, on the instrument system during a procedure may he usedto represent the shape of the instrument, particularly if an anatomicpassageway is generally static. Alternatively, a wireless device withposition or orientation controlled by an external magnetic field may beused for shape sensing. The history of the wireless device's positionmay he used to determine a shape for the navigated passageways.

The medical instrument system may, optionally, include a position sensorsystem 220. The position sensor system 220 may be a component of an EMsensor system with the sensor 220 including one or more conductive coilsthat may be subjected to an externally generated electromagnetic field.Each coil of the EM sensor system 220 then produces an inducedelectrical signal having characteristics that depend on the position andorientation of the coil relative to the externally generatedelectromagnetic field. In one embodiment, the EM sensor system may heconfigured and positioned to measure six degrees of freedom, e.g., threeposition coordinates X, Y, Z and three orientation angles indicatingpitch, yaw, and roll of a base point or five degrees of freedom, e.g.,three position coordinates X, Y, Z and two orientation angles indicatingpitch and yaw of a base point. Further description of an EM sensorsystem is provided in U.S. Pat. No. 6,380,732 (filed Aug. 11, 1999)(disclosing “Six-Degree of Freedom Tracking System Having a PassiveTransponder on the Object Being Tracked”), which is incorporated byreference herein in its entirety. In some embodiments, the shape sensormay also function as the position sensor because the shape of the sensortogether with information about the location of the base of the shapesensor (in the fixed coordinate system of the patient) allows thelocation of various points along the shape sensor, including the distaltip, to be calculated.

A tracking system 230 may include the position sensor system 220 and ashape sensor system 222 for determining the position, orientation,speed, pose, and/or shape of the distal end 218 and of one or moresegments 224 along the instrument 200. The tracking system 230 may beimplemented as hardware, firmware, software or a combination thereofwhich interact with or are otherwise executed by one or more computerprocessors, which may include the processors of a control system 116.

The flexible catheter body 216 includes a channel 221 sized and shapedto receive a medical instrument 226. Medical instruments may include,for example, image capture probes, biopsy instruments, laser ablationfibers, or other surgical, diagnostic, or therapeutic tools. Medicaltools may include end effectors having a single working member such as ascalpel, a blunt blade, an optical fiber, or an electrode. Other endeffectors may include, for example, forceps, graspers, scissors, or clipappliers. Examples of electrically activated end effectors includeelectrosurgical electrodes, transducers, sensors, and the like. Invarious embodiments, the medical tool 226 may be an image capture probethat includes a distal portion with a stereoscopic or monoscopic cameraat or near the distal end 218 of the flexible catheter body 216 forcapturing images (including video images) that are processed by avisualization system 231 for display. The image capture probe mayinclude a cable coupled to the camera for transmitting the capturedimage data. Alternatively, the image capture instrument may be afiber-optic bundle, such as a fiberscope, that couples to thevisualization system. The image capture instrument may be single ormulti-spectral, for example capturing image data in one or more of thevisible, infrared, or ultraviolet spectrums.

The medical instrument 226 may house cables, linkages, or otheractuation controls (not shown) that extend between the proximal anddistal ends of the instrument to controllably bend the distal end of theinstrument. Steerable instruments are described in detail in U.S. Pat.No. 7,316,681 (filed on Oct. 4, 2005) (disclosing “Articulated. SurgicalInstrument for Performing Minimally invasive Surgery with EnhancedDexterity and Sensitivity”) and U.S. Patent Application No. 12/286,644(filed Sept. 30, 2008) (disclosing “Passive Preload and Capstan Drivefor Surgical Instruments”), which are incorporated by reference hereinin their entireties.

The flexible catheter body 216 may also houses cables, linkages, orother steering controls (not shown) that extend between the housing 204and the distal end 218 to controllably bend the distal end 218 as shown,for example, by the broken dashed line depictions 219 of the distal end.Steerable catheters are described in detail in U.S. patent applicationSer. No. 13/274,208 (filed Oct. 14, 2011) (disclosing “Catheter withRemovable Vision Probe”), which is incorporated by reference herein inits entirety. In embodiments in which the instrument system 200 isactuated by a teleoperational assembly, the housing 204 may includedrive inputs that removably couple to and receive power from motorizeddrive elements of the teleoperational assembly. In embodiments in whichthe instrument system 200 is manually operated, the housing, 204 mayinclude gripping features, manual actuators, or other components formanually controlling the motion of the instrument system. The cathetersystem may he steerable or, alternatively, the system may benon-steerable with no integrated mechanism for operator control of theinstrument bending. Also or alternatively, one or more lumens, throughwhich medical instruments can be deployed and. used at a target surgicallocation, are defined in the walls of the flexible body 216.

In various embodiments, the medical instrument system 200 may include aflexible bronchial instrument, such as a bronchoscope or bronchialcatheter, for use in examination, diagnosis, biopsy, or treatment of alung. The system 200 is also suited for navigation and treatment ofother tissues, via natural or surgically created connected passageways,in any of a variety of anatomic systems, including the colon, theintestines, the kidneys and kidney calices, the brain, the heart, thecirculatory system including vasculature, and the like.

The information from the tracking system 230 may be sent to a navigationsystem 232 where it is combined with information from the visualizationsystem 231 and/or the preoperatively obtained models to provide thesurgeon or other operator with real-time position information on thedisplay system 110 for use in the control of the instrument 200. Thecontrol system 116 may utilize the position information as feedback forpositioning the instrument 200. Various systems for using fiber opticsensors to register and display a surgical instrument with surgicalimages are provided in U.S. patent application Ser. No. 13/107,562,filed May 13, 2011, disclosing, “Medical System Providing DynamicRegistration of a Model of an Anatomic Structure for Image-GuidedSurgery,” which is incorporated by reference herein in its entirety.

In the embodiment of FIG. 2A. the instrument 200 is teleoperated withinthe teleoperational medical system 100. In an alternative embodiment,the teleoperational assembly 102 may be replaced by direct operatorcontrol. In the. direct operation alternative, various handles andoperator interfaces may be included for hand-held operation of theinstrument.

In alternative embodiments, the teleoperated system may include morethan one slave manipulator assembly and/or more than one masterassembly. The exact number of manipulator assemblies will depend on themedical procedure and the space constraints within the operating room,among other factors. The master assemblies may be collocated, or theymay be positioned in separate locations. Multiple master assembliesallow more than one operator to control one or more slave manipulatorassemblies in various combinations.

As shown in greater detail in FIG. 2B, medical tool(s) 228 for suchprocedures as surgery, biopsy, ablation, illumination, irrigation, orsuction can be deployed through the channel 221 of the flexible body 216and used at a target location within the anatomy. If, for example, thetool 228 is a biopsy instrument, it may be used to remove sample tissueor a sampling of cells from a target anatomic location. The medical tool228 may be used with an image capture probe also within the flexiblebody 216. Alternatively, the tool 228 may itself be the image captureprobe. The tool 228 may be advanced from the opening of the channel 221to perform the procedure and then retracted back into the channel whenthe procedure is complete. The medical tool 228 may be removed from theproximal end 217 of the catheter flexible body or from another optionalinstrument port (not shown) along the flexible body.

FIG. 3 illustrates the catheter system 202 positioned within an anatomicpassageway of a patient anatomy. In this embodiment, the anatomicpassageway is an airway of human lungs 201. In alternative embodiments,the catheter system 202 may be used in other passageways of an anatomy.

FIG. 4 is a flowchart illustrating a general method 450 for use in animage-guided surgical procedure. At a process 452, pre--operative orintra-operative image data is obtained from imaging technology such as,computed tomography (CT), magnetic resonance imaging (MRI), fluoroscopy,thermography, ultrasound, optical coherence tomography (OCT), thermalimaging, impedance imaging, laser imaging, or nanotube X-ray imaging.The pre-operative or intra-operative image data may correspond totwo-dimensional, three-dimensional, or four-dimensional (including e.g.,time based or velocity based information) images. For example, the imagedata may represent the human lungs 201 of FIG. 3. At a process 454,computer software alone or in combination with manual input is used toconvert the recorded images into a segmented two-dimensional orthree-dimensional composite representation or model of a partial or anentire anatomic organ or anatomic region. The composite representationand the image data set describe the various locations and shapes of thepassageways and their connectivity. More specifically, during thesegmentation process the images are partitioned into segments orelements (e.g., pixels or voxels) that share certain characteristics orcomputed properties such as color, density, intensity, and texture. Thissegmentation process results in a two- or three-dimensionalreconstruction that forms a model of the target anatomy based on theobtained image. To represent the model, the segmentation process maydelineate sets of voxels representing the target anatomy and then applya function, such as marching cube function, to generate a 3D surfacethat encloses the voxels. The model may be made by generating a mesh,volume, or voxel map. Additionally or alternatively, the model mayinclude a centerline model that includes a set of interconnected linesegments or points extending through the centers of the modeledpassageways. Where the model includes a centerline model including a setof interconnected line segments, those line segments may be converted toa cloud or set of points. By converting the line segments, a desiredquantity of points corresponding to the interconnected line segments canhe selected manually or automatically. At a process 456, the anatomicmodel data is registered to the patient anatomy prior to and/or duringthe course of an image-guided surgical procedure on the patient.Generally, registration involves the matching of measured point topoints of the model through the use of rigid and/or non-rigidtransforms, Measured points may be generated using landmarks in theanatomy, electromagnetic coils scanned and tracked during the procedure,or a shape sensor system. The measured points may be generated for usein an iterative closest point (ICP) technique described in detail atFIG. 6 and elsewhere in this disclosure. Other point set registrationmethods may also be used in registration processes within the scope ofthis disclosure.

Other registration methods for use with image-guided surgery ofteninvolve the use of technologies based on electromagnetic or impedancesensing. Metallic objects or certain electronic devices used in thesurgical environment may create disturbances that impair the quality ofthe sensed data. Other methods of registration may obstruct the clinicalworkflow. The systems and methods described below perform registrationbased upon ICP, or another point set registration algorithm, and thecalibrated movement of a point gathering instrument with a fiber opticshape sensor, thus eliminating or minimizing disruptions in the surgicalenvironment. Other registration techniques may be used to register a setof measured points to a pre-operative model or a model obtained usinganother modality. In the embodiments described below, EM sensors on thepatient and the instrument and optical tracking systems for theinstrument may be eliminated.

FIGS, 5A, 5B, and 5C illustrate some of the steps of the general method450 illustrated in FIG. 4. FIG. 5A illustrates a segmented model 502 ofa set of anatomic passageways created from pre-operative orintra--operative imaging data. In this embodiment, the passageways areairways of a human lung. Due to naturally occurring limitations or tolimitations set by an operator, the segmented model 502 may not includeall of the passageways present within the human lungs. For example,relatively narrow and/or distal passageways of the lungs may not hefully included in the segmented model 502. The segment model 502 may bea three-dimensional model, such as a mesh model, that including thewalls defining the interior lumens or passageways of the lungs.

Based on the segmented model 502, a centerline segmented model 504 maybe generated as shown in FIG. 5B. The centerline segmented model 504 mayinclude a set of three-dimensional straight lines or a set of curvedlines that correspond to the approximate center of the passagewayscontained in the segmented model 502. The higher the resolution of themodel, the more accurately the set of straight or curved lines willcorrespond to the center of the passageways. Representing the lungs withthe centerline segmented model 504 may provide a smaller set of datathat is more efficiently processed by one or more processors orprocessing cores than the data set of the segmented model 502, whichrepresents the walls of the passageways. In this way the functioning ofthe control system 112 may be improved. As shown in FIG. 5B, thecenterline segmented model 504 includes several branch points, some ofwhich are highlighted for visibility in FIG. 5B. The branch points A, B,C, D, and E are shown at each of several of the branch points. Thebranch point A may represent the point in the model at which the tracheadivides into the left and right principal bronchi. The right principalbronchus may be identified in the centerline segment model 504 as beinglocated between branch points A and B. Similarly, secondary bronchi areidentified by the branch points B and C and between the branch points Band E. Another generation may be defined between branch points C and D.Each of these generations may be associated with a representation of thediameter of the lumen of the corresponding passageway. In someembodiments, the centerline model 504 may include an average diametervalue of each segmented generation. The average diameter value may be apatient-specific value or a more general value derived from multiplepatients.

In other embodiments, the segemented model 502 may be used to producethe centerline segment 504 or another suitable model including a cloud,set, or collection of points as follows. When the segmented model 502comprises a mesh representing the internal surfaces of one or morepassageways, a subset of vertices of a mesh as represented in a storeddata file including the model 502 may be used. Alternatively, ageometric center of voxels that represent volumes or the passageway's inthe segmented model 502 may be used. Additionally, combinations ofvarious approaches may be used to generate a first set of points, suchas the centerline segment model 504. For example, a subset of verticesof the mesh may be used along with the geometric center of voxels fromthe model.

In some embodiments, the centerline segmented model 504 is representedin data as a cloud., set, or collection of points in three-dimensionalspace, rather than as continuous lines. FIG. 5C illustrates thecenterline segmented model 504 as a set of points 506. In data, each ofthe points of the set of model points may include coordinates such as aset of X_(M), Y_(M), and Z_(M), coordinates, or other coordinates thatidentify the location of each point in the three-dimensional space. Insome embodiments, each of the points may include a generation identifierthat identifies which passageway generation the points are associatedwith and/or a diameter or radius value associated with that portion ofthe centerline segmented model 504. in some embodiments, informationdescribing the radius or diameter associated with a given point may beprovided as part of a separate data set.

After the centerline segmented model 504 is generated and stored in dataas the set of points 506 shown in FIG. 5C, the centerline segmentedmodel 504 may be retrieved from data storage for use in an image-guidedsurgical procedure. In order to use the centerline segmented model 504in the image-guided surgical procedure, the model 504 may be registeredto associate. the modeled passageways in the model 504 with thepatient's actual anatomy as present in a surgical environment. Use ofthe model 504 in point set registration includes using the set of points506 from the model 504.

FIGS. 6A and 6B illustrate an exemplary surgical environment 600according to some embodiments, with a surgical coordinate system X_(S),Y_(S), Z_(S), in which a patient P is positioned on a platform 602. Thepatient P may be stationary within the surgical environment in the sensethat gross patient movement is limited by sedation, restraint, or othermeans. Cyclic anatomic motion including respiration and cardiac motionof the patient P may continue, unless the patient is asked to hold hisor her breath to temporarily suspend respiratory motion. Accordingly,data may be gathered at a specific, direction phase in respiration, andtagged and identified with that phase, in some embodiments. In otherembodiments, the phase during which data is collected may be. inferredfrom physiological information collected from the patient. Within thesurgical environment 600, a point gathering instrument 604 is coupled toan instrument carriage 606. In various embodiments, the point gatheringinstrument 604 may use EM sensors, shape-sensors, and/or other sensormodalities. The instrument carriage 606 is mounted to an insertion stage608 fixed within the surgical environment 600. Alternatively, theinsertion stage 608 may he movable but have a known location (e.g., viaa tracking sensor or other tracking device) within the surgicalcoordinate system. The instrument carriage 606 may he a component of ateleoperational manipulator assembly (e.g., assembly 102) that couplesto the instrument 604 to control insertion motion (i.e., motion in anX_(S) direction) and, optionally, motion of a distal end of theinstrument in multiple directions including yaw, pitch, and roll. Theinstrument carriage 606 or the insertion stage 608 may includeservomotors (not shown) that control motion of the instrument carriagealong the insertion stage.

The point gathering instrument 604 may include a flexible catheter 610coupled to a proximal rigid instrument body 612. The rigid instrumentbody 612 is coupled and fixed relative to the instrument carriage 606.In the illustrated embodiment, an optical fiber shape sensor 614 isfixed at a proximal point 616 on the rigid instrument body 612. In analternative embodiment, the point 616 of the sensor 614 may be movablealong the body 612 but the location of the point may be known (e.g., viaa tracking sensor or other tracking device). The shape sensor 614measures a shape from the point 616 to another point such as the distalend 618 of the catheter 610. The point gathering instrument 604 may hesubstantially similar to the medical instrument system 200.

A position measuring device 620 provides information about the positionof the rigid instrument body 612 as it moves on the insertion stage 608along an insertion axis A. The position measuring device 620 may includeresolvers, encoders, potentiometers, and other mechanisms that determinethe rotation and orientation of the motor shafts controlling the motionof the instrument carriage 606 and consequently the motion of therigidly attached instrument body 612. In this embodiment, the insertionstage 608 is linear, but in alternative embodiments it may be curved orhave a combination of curved and linear sections. Optionally, the lineartrack may be collapsible as described, for example, in U.S. ProvisionalPatent Application No. 62/029,917 (filed Jul. 28, 2014)(disclosing“Guide Apparatus For Delivery Of A Flexible Instrument And Methods OfUse”) which is incorporated by reference herein in its entirety. FIG. 6Ashows the instrument body 612 and carriage 606 in a retracted positionalong the insertion stage 608. In this retracted position, the proximalpoint 616 is at a position L₀ on the axis A. In this position along theinsertion stage 608 a.n X_(S) component of the location of the point 616may he set to a zero or original value, With this retracted position ofthe instrument body 612 and carriage 606, the distal end 618 of thecatheter may he positioned just inside an entry orifice of the patientP. Also in this position, the position measuring device may be set to azero or original value (e,g, I=0). In FIG. 6B, the instrument body 612and the carriage 606 have advanced along the linear track of theinsertion stage 608 and the distal end of the catheter 610 has advancedinto the patient P. In this advanced position, the proximal point 616 isat a position L₁ on the axis A.

Embodiments of the point gathering instrument 604 may collect measuredpoints using any number of modalities, including EM sensing andshape-sensing. As the measurement points are collected from within thepassageways of a patient, the points are stored in a data storagedevice, such as a memory. The set of measured points may he stored in adatabase that includes at least some, but may include all, of themeasured points obtained during the procedure or immediately before theprocedure. As stored in memory, each of the points may be represented bydata comprising coordinates of the point, a timestamp, and a relativesensor position or individual sensor ID (when multiple sensorsdistributed along a length of the point gathering instrument 604 areused to determine the location of several points simultaneously). Insome embodiments, data representing each point may also include arespiratory phase marker that indicates the respiratory phase of thepatient in which the point was collected.

FIG. 7 is a flowchart illustrating a method 700 used to provide guidanceto a clinician in an image-guided surgical procedure on the patient P inthe surgical environment 600, according to an embodiment of the presentdisclosure. The method 700 is illustrated in FIG. 7 as a set of blocks,steps, operations, or processes. Not all of the illustrated, enumeratedoperations may be performed in all embodiments of the method 700.Additionally, some additional operations that are not expresslyillustrated in FIG-. 7 may be included before, after, in between, or aspart of the enumerated processes. Some embodiments of the method 700include instructions corresponded to the processes of the method 700 asstored in a memory. These instructions may he executed by a processorlike a processor of the control system 112,

Thus, some embodiments of the method 700 may begin at a process 702, inwhich a calibration procedure is performed to calibrate, with a positionmeasuring device like the point gathering instrument 604 or anothersuitable device, a relative position and/or orientation of a sensorreference point along an insertion path. For example, the pointgathering instrument 604 of FIGS. 6A and 613 may he used to determine aposition and orientation of the point 616 as the carriage 606 moves froma retracted position with the point 616 at location L₀ to an advancedposition with the point 616 at the location L₁. The calibrationprocedure determines the direction of the movement of the point 616 foreach change in the position measuring device 620. In this embodiment,where the insertion stage 608 restricts movement of the carriage 606 toa linear path, the calibration procedure determines the direction of thestraight line. From the slope of the insertion stage track, the positionand orientation of the point 616 in the surgical environment 600 may bedetermined for every corresponding measurement of the position measuringdevice 620. In an alternative embodiment, if the insertion stage has acurved or otherwise non-linear shape, the calibration procedure maydetermine the non-linear shape so that for every measurement of theposition device, the position and orientation of the point 616 in thesurgical environment may be determined. For example, the distal tip ofthe catheter may be held in a fixed position while the instrument bodyis routed along the non-linear insertion stage. The position andorientation data collected by the shape sensor from the fixed point 616is correlated. with the position measuring device data as the instrumentbody is routed along the insertion stage, thus calibrating movement ofthe point 616 along the axis A of the insertion stage 608.

At a process 704, the distal end 618 of the catheter 610 traverses thepatient P's anatomic passageways (e.g., airways of the patient's lungs)recording, via data from the shape sensor 614, or another sensor such asan EM sensor or sensors provided on the catheter 610, location data forthe distal end of the catheter and/or other points along the shape ofthe shape sensor. This location data may include, or be processed toobtain, a set of measured points as described herein. More specifically.the movement of the distal tip of the catheter 610 may be controlled viateleoperational, manual, or automated control (e.g., via master assembly106) to survey a portion of the anatomic passageways.

For example, teleoperational control signals may cause the carriage 606to move along the axis A, causing the distal tip 618 of the catheter toadvance or retract within the anatomic passageways. Also oralternatively, teleoperational control signals may cause actuation ofcontrol members extending within the surgical instrument to move thedistal tip 618 in a range of movements including yaw, pitch, and roll.As the catheter is moved within the plurality of passageways. shapesensor data (and/or other position data in other embodiments that do notinclude a sensor sensor) is gathered for multiple locations of thedistal tip. In some embodiments, the catheter may extend up toapproximately three inches into the various passageways. In someembodiments, the catheter may he extended through or into approximatelythree branched generations on each side of the lung. The number ofgenerations accessible with the catheter 610 may increase as thediameter of the. flexible catheter 610 decreases and/or the flexibilityof the flexible catheter 610 increases.

With reference to FIG. 8, shape sensor data is gathered for a set ofmeasured data points D, in some embodiments. These measured points maybe stored in memory as data sets or point pools with coordinates,timestamps, sensor IDs, respiration phase information, etc., for eachgathered point, as is discussed herein in connection with FIG. 13. Thiscollected set of spatial information provided by data points D from theshape sensor or other point collection device may be gathered as thedistal end 618 of the catheter 610 is moved to a plurality of locationswithin the surgical space 600. The location of a given collected datapoint D_(X) in the surgical environment space 600 is determined bycombining information from the position measuring device 620 when thedistal end of the catheter is located at the point D_(X) with the shapedata from the shape sensor when the distal end of the catheter islocated at the point D_(X). Points may also be collected along thelength of the catheter. In both such cases, the data from the positionmeasuring device 620 and the calibrated path of the fixed sensor point616 provides the position of the sensor point 616 in the patientsurgical environment 600 when the distal end 618 of the catheter is atthe point D_(X). For example, encoder data from one or more motorscontrolling movement of the carriage 606 along the track 608 and thecalibration data from the movement of the carriage along the trackprovides the position of the sensor point 616 in the surgicalenvironment 600 when the distal end of the catheter is at the pointD_(X). The shape sensor provides the shape of the instrument between thefixed sensor point 616 and the distal end 618. Thus, the location of thepoint D (where the distal end 618 is located) in the surgicalenvironment space 600 can he determined from the calibrated positionmeasuring data and the shape sensor data recorded when the distal end isat point D_(X) and one or more points along the. length of the shapesensor may be used to determine other points D as well, The location in.the surgical environment 600 coordinate space for all of the data pointsD in the set of gathered data points (i.e. calibrated position of theproximal point 616 together combined with the shape sensor data for thelocation of the distal end 618 relative to the point 616) is a referenceset of spatial information for the instrument that can be registeredwith anatomic model information.

Referring again to FIG. 7, at a process 706, one or more of the gathereddata points D may correspond to landmark locations in the patientanatomy. In some embodiments, the gathered data points D that correspondto landmarks may be used to seed a registration process, such as an TCPprocess. In some embodiments of the method 700, the process 706 may nothe performed and an automatic seed generation algorithm may be used inplace of the process 706. Additionally, some embodiments of ICPprocesses may not include a seeding process. This subset of gathereddata points D that correspond to one or more landmarks may be referredto as seed points. The data representing the subset of gathered datapoints D that correspond to landmarks may include a landmark indicatorwhen stored in memory. With reference to FIG. 9, a set of anatomicpassageways 900 includes main carinas C₁, C₂, C₃ where the passageways900 fork. A data point D can be gathered for the location of each carinaby moving the distal end of the catheter to the respective carinalocations. For example, a data point D_(L1) can be gathered at thecarina C₁. A data point D_(L2) can be gathered at the carina C₂. A datapoint D_(L3) can be gathered at the carina C₃, The carinas or othersuitable landmarks can be located in the patient surgical environment600 as described above for point D_(X).

Referring again to the method 700 of FIG. 7, at a process 708 anatomicmodel information is received. The anatomic model information may be thesegmented centerline model 504 as described in FIG. 5C. Referring againto FIG. 9, the anatomic model information may be represented as acenterline model 550 of branched anatomic passageways. In someembodiments, the model may include one or more landmark points to matchto the seed points D_(L1), D_(L2), and D_(L3). These points included inthe model to match to the seed points D_(LI), D_(L), and D_(L3) may notbe centerline points in some embodiments, but may be included in thecenterline model 550 to facilitate seeding of a subsequent registrationprocess. In some embodiments, the centerline model 550 may include moremodel landmark points than M_(L1), M_(L2), and M_(L3).

Referring again to FIG. 7, at a process 712 registration of the anatomicmodel information 550 with the set of gathered data points D from thesurgical environment 600 is performed. Registration may be accomplishedusing a point set registration algorithm such as an iterative closestpoint (ICP) technique as described in processes 512-520, or byimplementation of another registration algorithm. Prior to the process712, at process 710, the ICP registration is seeded with knowninformation about the displacement and orientation relationship betweenthe patient surgical environment and the anatomic model. In thisembodiment (FIG. 9), for example, the carina landmarks C₁, C₂, C₃ areidentified in the anatomic model information as points M_(L1), M_(L2),M_(L3). In alternative embodiments, the anatomic model information maybe represented in other ways, e.g. as centerline segments or axes of a3D mesh model. The recorded landmark data points D_(L1), D_(L2), andD_(L3) from the patient surgical environment are each matched to acorresponding model points M_(L1), M_(L2), and M_(L3). (i.e., D_(L1)matches to M_(L1), etc.) With the points matched, an initial transform(e.g., change in position and/or orientation) between landmark datapoints D_(L1), D_(L2), and D_(L3) arid model points M_(L1), M_(L2), andM_(L3) is applied, at a process 711. This initial transform determinedbased on the landmark data points D_(L1), D_(L2), D_(L3) and the modellandmark points M_(L1), M_(L2), M_(L3) may be applied to all of thegathered data points D. This seeding process, based on a few landmarkpoints, provides an initial coarse registration of the gathered datapoints D to the anatomic model information 550.

The initial transform may be a rigid transform in which all landmarkdata points are transformed by the same change in position andorientation or may be a non-rigid transform in which the landmark datapoints are transformed by different changes in position and orientation.In some embodiments, in which the set of measured data points Dundergoes rotation about at least one axis, the motion of eachindividual point may vary due to the placement of the axis or axes ofrotation of the set of data points D.

The initial transformation performed at the seed stage can be performedbased on many different model points and many different measured points.Similarly, the measured points used in the seed stage can be provided inmany different ways. In some embodiments, multiple methods of seeding toproviding the initial transformation are performed. and each is checkedfor error. The method that provides the smallest error between theanatomic information model 550 and the set of points D may be used tobegin the registration process.

For example, the major “Y” formation provided by the trachea and theleft and right main bronchii shown in FIG. 9 may be used as explainedherein. Each of the first three main bifurcation points or carinas C₁,C₂, and C₃ may be identified in the model and in the set of data pointsD. In some embodiments, the process of collecting data points D maycause an identifier to he included as part of the recorded data forcertain points to be used in the seeding process to be included in datafor that point. This identifier may be included automatically as part ofa workflow or a clinician controlling the point gathering instrument 604may use an interface to manually indicate, by clicking a button forexample, that a point defined by the distal end 618 of the catheter 610point gathering instrument 604 at a specific time is to he used in theseeding process (i.e., is a landmark data point). For example, afterinserting the catheter the clinician may position the distal end 618 atthe carina C₂ and click a button to capture the location as a datapoint. D_(L2). The clinician may then navigate, in some embodimentsusing an image sensor positioned within the passageway's of the patientP. to the carina C₁ and click the button to capture the location as thedata point D_(L1). The clinician may then navigate to the carina C₃ andclick the button to capture the location as (he data point. D_(L3). Insome embodiments, the interface of a medical system may direct theclinician to navigate to a specific location and then affirm through themanipulation of an interface element that the distal end 618 is at thespecific location, at which point the corresponding data point iscollected. The data points gathered in this way may include coordinatesand other information described herein and may further include anindication that the data points are seed points or landmark points thatcan be used in the seeding process to perform the initial coarsetransformation.

Other user interface interactions may be used to trigger the collectionof a data point D_(X) or data points D_(X) for use in the seedingprocess. For example, where the medical system includes a voicerecognition component, the clinician may speak aloud to identify apresent location of the distal end 618 as a seed point or to confirm acollected point as a seed point corresponding to a specific locationrequested by a workflow. While in the example above a workflow isprovided by the medical system to collect the data points D_(L1),D_(L2), D_(L3) for use in the seeding process, in other embodiments auser may pick preferred landmarks, including landmarks other than themain carinas as shown in FIG. 9. Through the user interface, theclinician may identify the anatomic location of the selected landmarksso that a corresponding location in the anatomic model information maybe approximated.

In some embodiments, sensors provided at the distal or proximal end ofthe catheter 610 may he used to trigger the collection of data pointsincluding data points D_(L1), D_(L2), and D_(L3) for use in the seedingprocess. For example, as part of a workflow the medical system may usethe first three collected data points D_(X1), D_(X2), and D_(X3). Insuch an embodiment, the clinician may navigate to the main carina C₁ andcause physical contact between the main carina C₁ and the distal end618. A torque sensor or an encoder for actuator controlling the distalend of the catheter may register resistance or a force against thedistal end and trigger the collection of the data point D_(L1) inresponse.

In some embodiments, a touch sensor such as a capacitive or Hall effectsensor may he positioned along the flexible instrument to provide anindication when the instrument is close to or in contact with a wall ofthe anatomic passageway. Thus, the touch sensor may provide informationabout the. shape and size of passageways that may he used to identifycorresponding characteristics in the model data.

While the main carinas may be used in the seeding process for sonicembodiments, other embodiments may rely on other anatomic features toperform the. initial registration. For example, when the distal end 618of the catheter 610 is passed through an endotracheal (EY) tube used toguide the catheter 610 through the mouth of the patient P the catheter610 may conform to a known bend or curve corresponding to theendotracheal tube. An exemplary ET tube 622 is illustrated in FIGS, 6Cand 6D, Even if a bend 621 in the tube is not precisely known, thecurvature may be sufficiently distinctive to be identified ascorresponding to the upper respirator); track and trachea because theportion of the catheter 610 at the proximal end of the ET tube 622 formsa nearly 90° angle with respect to the portion of the catheter 610 atthe distal end of the ET tube 622. The pose of the proximal end of thepoint gathering instrument may be known due to sensors in the joints ofthe teleoperatiotial assembly 102. Based on this pose information and acurve of the endotracheal tube, which may be easily identified usingshape sensor data, the trachea of the patient P may be identified andused to seed an initial transformation. While the orientation of thepatient P to the point gathering instrument 604 may already he known, bynavigating the distal end 618 into either the left or right primarybronchus (shown in FIG. 6C) additional information may be gathered thatcan be used to seed the initial transformation. For example, using shapesensor or EM sensor data, information characterizing a first roughlyright angle may be collected between the entrance of the ET tube andexit of the ET tube into the length of the trachea of the patient P.This first angle may identify a plane that may he expected to bisect theanatomic model information 550. When the distal end 618 transitions fromthe trachea into either the left or right primary bronchus, a secondangle defining a second plane may be identified, The first and secondplaces are roughly orthogonal. By using the first and second angles, themedical system may identify a right-left orientation of the patientwhich may be used to seed the registration process by roughly orientingthe data points D with the anatomic model information 550.

Referring again to FIG. 6C, shown therein is a close-up view of thepatient P as shown in the FIGS. 6A and 6B. FIG. 6C illustrates an ETtube 622 positioned within the patient P to guide the catheter 610 intothe patient P's lungs. The ET tube 622 includes an interior surface 623that may include a distinctive color and/or a distinctive pattern.

In embodiments in which an endoscopic camera is incorporated into orused in conjunction with the point gathering instrument 604, imageinformation may also be used to provide a seed. For example, image datatray be used by the medical system to determine whether or not thecatheter 610 is in the trachea, or another passageway, of the patient P.This may be done, for example, by using a camera to monitor for a changein anatomic color, texture, or anatomic feature associated with changesin anatomic region (e.g., the entrance of to the trachea).Alternatively, the camera may be used to monitor for a change in coloror pattern of the interior surface 623 of the ET tube 622 or themovement of the camera past a distal end of the ET tube which terminatesin the trachea.

To facilitate the detection of the catheter 610 entering and/or exitingthe ET tube, the ET tube may include a distinguishing color, marking,and/or pattern. For example, the ET tube may be bright green, orange, oranother color. In some embodiments, the ET tube may include markingssuch as symbols or alphanumeric characters. In some embodiments, the ETtube may include a pattern such as a striped pattern alternating betweena bright color and dark color. Additionally, in some embodiments, the ETtube 622 may be coated with a reflective coating. Based on the color,marking, and/or pattern of ET tube, the medical system may use imagesobtained by the camera to determine whether the distal end 618 is withinthe ET tube or has exited into the trachea. Examples of the interiorsurface 623 are illustrated in FIG. 6D. The colors. markings, and/orpatterns may also be captured in the pre-operative or intra-operativeimaging data and thus serve as fiducial markers in the seedingprocedure.

As shown in FIG. 6D, the interior surface 623A has a solid color. Thesolid color may he a hunter orange, a fluorescent green, or any colorthat is readily distinct to machine vision when compared with thenaturally occurring colors of the passageways of the patient P. Theinterior surface 623B shows a pattern including alternating stripes ofat least two colors. For example, black and white stripes may be used insome embodiments, While the stripes of the interior surface 623B are allequal, the stripes of the interior surface 623C may include stripes ofdifferent widths. As shown the light stripes increase in width, whilethe dark stripes have a common width. In some embodiment both the lightand dark stripes may vary in width along the length of the ET tube 622.While the stripes of interior surface 623B and 623C are arrangedorthogonally to a central axis of the ET tube 622, the interior surface623D includes stripes that are not orthogonal to the central axis. Thesurface 623D includes alternating strips oblique to the central axis. Inalternative embodiments, the colors, markings, or patterns may changealong the length of the surface 623 to provide an indication of theinsertion depth of the camera.

In other embodiments the pose of the catheter 610 may be estimated basedon an endoscopic image by comparing an expected position to an actualimage. For example, a camera. image of the main carina during insertionof a medical instrument, such as an endoscope, may he compared to avirtual segmented representation of the main carina and used to estimateinsertion depth and roll angle of the endoscope. The roll angle may havetwo possible solutions 180° apart corresponding to the left and rightmain bronchii. of the patient. In. such an instance, two registrationprocesses could be initiated, one for each of the possible solutions.The registration that provides the better result could be maintained andcontinued while the other registration could be discarded.

Referring again to the method 700 of FIG. 7, at a process 714 with theinitial coarse transformation performed to initiate the registrationprocess, the set of measured data points D gathered from within thepatient P is matched to the anatomic model information 550 (FIG. 10).For example, each of the measured data points D may be matched with theclosest point in the anatomic model information 550. In this embodiment,the anatomic model information 550 is a set of points along thecenterlines generated from a three-dimensional anatomic model, like thesegmented centerline model 504 is generated from the segmented model 502of FIG. 5C. The registration algorithm identifies matches betweenclosest points in the gathered data points D and in the set of anatomicmodel points. In various alternatives, matching may be accomplished byusing brute force techniques, KD tree techniques, etc. Some matches maybe discarded based on maximum distance threshold calculations, maximumangle threshold calculations, or other metrics employed to filter outmatches that are not deemed to he reliable enough for inclusion in themodel, The anatomic model points may he represented by any of severaldifferent kinds of points, including centerline points, mesh points,and/or volume points. In some embodiments, only a subset of measureddata points D are matched with the set of points in the anatomic model.Multiple heuristics may be implemented to determine which of themeasured data points D are included in the subset of measured points.

At a process 716, the motion needed to move the set of gathered datapoints D to the position and orientation of the matched anatomic modelpoints of the anatomic model information 550 is determined. Morespecifically, an overall computed offset in position and orientation isdetermined for the set of gathered data points D. FIG. 8, for example,illustrates an initial offset of approximately 20° in orientation and 40mm in displacement between the gathered data points D and the anatomicmodel information 550. In some embodiments, the computed correctivemotion may be limited such that only a certain number of degrees ofrotation for a certain number of millimeters of displacement may beapplied in a single iteration of the process 712. In some embodiments,even if a rotation or reorientation of the anatomic model information550 of 20° is computed, the medical system may limit the change inorientation to 10°, 5°, or less. Similarly, in some embodiments even ifa displacement of 40 mm is computed, the medical system may limit thedisplacement available in a single iteration to 20 mm, 10 mm, 5 mm, orless. In some embodiments, the limits may change according to a numberof iterations performed such that less movement is permitted in lateriterations than in earlier iterations.

At a process 718, the set of gathered data points D are transformedusing a rigid or non-rigid transformation that applies the computedoffset in displacement and orientation to move each point in the set ofgathered data points D. A limited computed offset may be applied if thecomputed offset is greater than establish limits. In an alternativeembodiment, the modeled data points may be transformed by using a rigidor non-rigid transform that applies the computed offset in displacementand orientation to move each point in the set of modeled data points 550toward the gathered data points D. Accordingly, some embodiments of thepresent disclosure may refer to registering measured points to modelpoints and moving (including translating and/or changing the orientationof) the measured points to better align with the model points. Theseembodiments also encompass registering measured points to model pointsand reorienting the model points to better align with the measuredpoints. In still another alternative embodiment, the computed offset maybe partially applied to the set of gathered data points D and partiallyapplied to the modeled data points 550 such that both sets of points aretransformed to converge in a common frame of reference distinct fromeither the frame of the gathered data points or the frame of the modeleddata points.

At a process 720, the convergence of the gathered data points D and thematched anatomic model points 550 is evaluated. In other words, errorfactors for orientation and displacement may be determined for eachmatched point set. If the error factors in aggregate are greater than athreshold value, additional iterations of processes 714-720 may berepeated until the overall position and orientation error factors fallsbelow the threshold value. A result of this process is illustrated inFIG. 10. The result shown in FIG. 10 may represent the result ofrepeating the processes 712 for multiple iterations. For example, morethan 50 iterations may be performed to converge on a satisfactoryregistration. However, fewer than 30 iterations may be needed to achievea satisfactory registration, in some embodiments. The registrationprocess including seeding and the processes 712 may be understood asorienting a set of anatomic model information 550 to a set of points Dpresent in space defined by a surgical environment; as orienting the setof points D to a set of anatomic model information 550 in an anatomicmodel space; or as orienting both sets of points D and anatomic modelpoints 550 to a common space distinct from the surgical environment orthe model space.

The sum of the computed motions required to minimize the error betweenthe set of measured points D and anatomic model information 550 may beapplied to a model having greater detail than is present in the anatomicmodel information 550. For example, after registering the segmentedcenterline model 504 of FIG. 5C, the same transformation may be appliedfor another model such as the segmented model 502 of FIG. 5A.Thereafter, a clinician may be presented with a user interface thatdisplays both of the segmented model 502 or portions thereof and liveinformation such as the current position of a catheter within thepassageway's of a long of patient P. The segmented model 502 and thelive information may be presented in separate windows or screens or maybe overlaid and presented jointly in a single window.

The registration process 712 may be recomputed multiple times during asurgical procedure (e.g., once every ten second, once every minute, onceevery five minutes, etc.) periodically and/or in response to deformationof the passageways caused by cyclic anatomic motion, instrument forces,and/or other changes in the patient environment or in the patient'sorientation to the environment, such as by a patient movement,

After the anatomic model is registered to the surgical environment, animage-guided surgical procedure may, optionally, be performed. Theanatomic model may include previously captured details from modalitiesthat are difficult to use during a surgical procedure and so aregenerally captured pre-operatively. Referring again to FIG. 8, atprocess 722, during a surgical procedure, a current location of asurgical instrument in the surgical environment is determined. Forexample, the position of the surgical instrument, and particularly thedistal end 618, in the surgical environment 600 may he determined usingposition sensors such as EM sensors. Alternatively, the data from theposition measuring device 620 and the calibrated path of the fixed.sensor point 616 provides the position of the sensor point 616 in thepatient surgical environment 600 when the catheter is in a currentlocation. The shape sensor or multiple discrete sensors provide theshape of the instrument between the fixed sensor point 616 and thedistal end 618, Thus, the current location of the catheter 610 and,particularly, the distal end 618 of the catheter in the surgicalenvironment space 600 can be determined from the calibrated positionmeasuring data and the shape sensor data.

At process 724, the previously determined registration transforms areapplied to the current instrument position and shape data to localizethe current instrument to the anatomic model. For example, the currentposition and orientation for the distal end of the instrument, datapoint D_(current) is transformed using the one or more transformiterations determined at process 712. Thus, the data point D_(current)in the surgical environment 600 is transformed to the anatomic modelspace. Alternatively, if the model data points have been transformed tothe surgical environment 600 and the catheter 610 is localized in thesurgical environment, process 724 may be omitted.

Optionally, the registration allows for the presentation of one or moreimages to assist with an image guided procedure. For example, an imageof the catheter superimposed on the segmented model passageways may bepresented. Additionally or alternatively, an internal image of theanatomic passageways within the model from the perspective of thelocalized distal end of the catheter (i.e. a view just distal of thedistal end 618) may be presented.

Referring now to FIGS. 11A-H, shown therein are several different rulesor heuristics that may be used in selecting a subset of measured datapoints D that are then used to match to an anatomic model to compute acorrective motion, By identifying points that include superiorinformation and ignoring points that include inferior information, acontrol system performing registration may operate more efficiently.Increased efficiency may permit. the periodic or event driven updatingof a registration or re-registration with minimal delay in thepresentation of images used for surgical navigation. in someembodiments, only a subset of measured data points is used in thematching process 714. A set of measured data points 1102 are shown asthey relate to an anatomic model 1104, illustrated as a collection ofsegments 1104A-G. While the segments I 104A-G of the anatomic model 1104are illustrated as lines in FIGS. 11A-H, these segments I 104A-G mayalso correspond to collections of model points (including points1106A-F) extending along the center of the passageways of a lung orother organ with passageways.

As shown in FIG. 11A, the set of measured data points 1102 exhibit ageneral correlation to the segments of the anatomic model 1104. Thiscorrelation may be a result of previous iterations of the registrationprocess or of the seeding of such registration process. When computing acorrective motion by which to transform the measured points 1102 tobetter align with the anatomic model 1104, multiple error values May beobtained as part of the process of computing the motion. In someembodiments, these error values may be distances between matched points.In general, the smallest error value associated with a specific measureddata point may be a match.

As illustrated in FIG. 11A, the measured data point 1102A matches to amodel point 1106A. A distance 1107 characterizes a distance or errorvalue associated with the matched pair of the measured data point 1102Aand the model point 1106A. These points may be matched because thedistance 1107 is the shortest distance between the measured data point1102A and any other model point in the anatomic model 1104. Othermatches illustrated in FIG. 11A include a match between the measuredpoint 1102B and model point 1106B, another match between the measuredpoint 11020 and model point 1106C, and another match between themeasured point 1102D and model point 1106D. The measured point 1102D isalso matched with the model point 1106E. The distance between themeasured point 1 102D and the model point 1106D and the distance betweenthe measured point 1102D and the model point 1106E may be identical ormay be substantially similar so as to be considered identical. Forexample, when the distances between a measured point and two modelpoints differ by less than a threshold percentage (e.g., 10%, 5%, 3%,etc.), the distances may be deemed identical for certain purposes. Insome embodiments, the distance between the model points 1106D and 1 106Emay be calculated, and, as a heuristic, both the matches may bediscarded when the distance between the model points is greater than thedistance between either of the model points and the measured point 1102Das this may be an indication that the two matches are associated withdifferent airways. When the distance between the model points is lessthan a distance between either of the model points and the measuredpoint. the matches may be averaged or a single match may be selected.When multiple measured points match to a single model point, themeasured points may he averaged to create a single averaged point.

In order to more efficiently and/or accurately register the measureddata points 1102 with the an-atomic model 1104, the effect of somepoints may be ignored based on one or more heuristics during thecomputation of the corrective motion. Matching saliency may he the basisfor ignoring or discarding a measured point. For example, because themeasured point 1102D is effectively matched to both the model points1106D and 1106E, the matches may be ignored when calculating thecorrective motion. This may be done by ignoring the measured point.1102D, When ignoring a measured point in the various algorithmsdisclosed herein, the measured point may be temporarily ignored. Forexample, the measured point 1102D and its associated matches with themodel points 1106D and 1106E may not be factored into the correctivemotion as determined in one iteration, but may be factored into thecorrective motions determined in subsequent iterations. Alternatively, ameasured point may be deleted permanently rather than temporarily.

The measured data point 1102C matches to the model point 11060. Themodel point 1106C is a “terminal point” in that it is the most distalpoint in the segment 1104G, a segment that does not connect to asubsequent segment at its distal end. In some embodiments, a heuristicprovides that measured points that match to a terminal point of themodel are ignored at least for one iteration. As shown in FIG. 11A, thematch between the point 1102C and the point 1106C may be ignored suchthat it is not factored into the computations that determine thecorrective motion to apply to the set of measured data points 1102 (orto the model 1104, depending on which set of points is being movedrelative to the other). Excluding measured points that match to aterminal point of the model may be done to avoid pulling the modeltoward unsegmented branches, because those measured points would likelyhave matched to a more distal passageway had that more distal passagewaybeen segmented to generate the model. In some embodiments, all measuredpoints that match to a terminal point on the anatomic model 1104 may beignored. The anatomic model 1104 used for registration may not representall of the passageways of the lung or other anatomy. Accordingly, themeasured points 1102 may he obtained from portions of the, lung that arenot included in the anatomic model 1104. The registration of themeasured points 1102 to the anatomic model 1104 may be performed moreefficiently and more accurately by the control system 112 by selectivelyignoring a subset of measured points that does not correspond well toany of the points of the anatomic model 1104. While points that areignored for one or more iterations may he considered in subsequentiterations in some embodiments, in other embodiments, measured pointsthat match to more than one model point or that match to a terminalmodel point may be deleted from the set of measured points.

Additionally, in some situations a “terminal point” may be a mostproximal point in a segment. For example, supposing segment 1104A to bethe segment of the model associated with the trachea, the most proximalpoint would correspond to the beginning of the trachea. Points thatcorrespond to the beginning of the trachea may be ignored for aniteration or more.

As another heuristic, timestamps associated with each measured point mayhe used. to determine which segment the measured point is associatedwith. For example a timestamp associated with measured point 1102D whenconsidered with the timestamps of other measured points recorded aroundthe same time, may indicate that the catheter was in the passagewayassociated with segment 1104D and could not have been in the passageway1104E at that time. Thus the temporal order of the measured points maybe used to determine whether a given measured point may be matched tomodel points.

FIG. 11B shows the set of measured points 1102 and the anatomic model1104 after a corrective motion is applied to transform the measuredpoints 1102 for a first iteration of the registration process. For asubsequent iteration of the registration process, the measured point1102A is now matched to a model point 1106F to which is it now closer.After the first iteration, the measured point 1102E is further away fromthe segment 1104A. For the first iteration, the measured point 1102Cmatched to the terminal model point I106C. For the subsequent iteration,the measured point 1102C matches to a different terminal model point,point 1106G.

FIG. 11C illustrates another heuristic by which the process 714 ofmatching measured data points to model points may he optimized toprovide more accurate and more efficient registration by the controlsystem 112. As shown in FIG. 11C, a measured point 1102F is matched to amodel point 1106H. Additional points of the measured points 1102 shownnear the measured point 1102F will also match to points on the modelsegment 1104G. However, because no measured points 1102 are matched tothe segment 1104C, the match between the measured point 1102F and themodel point 1106H may be ignored when computing a corrective motion.This may be done on the assumption that for a point to be associatedwith a more distal passageway (a higher generation passageway) ameasured point should be obtained that matches to the immediately prior(less distal) passageway. The immediately prior passageways may bereferred to as a “parent” while the immediately distal passageway may bereferred to as a “child.” Here, because no measured points match tosegment 1104C of the anatomic model 1104, the matches to the connecting,but more distal segment 1104E are suspect. In some embodiments, themeasured point 1102F may be ignored for a specified number ofiterations, e.g., one or more, or the measured point 1102F may bedeleted, permanently from the set of measured points 1102.

Also illustrated in FIG. 11C, two measured points 1102G and 1102H areshown matched to model points 11061 and 1106J, respectively. The modelpoints 1106I and 1106J are part of the model segment 1104E. In someembodiments, another heuristic may provide that a threshold number ofmeasured points is required to match to a specific model segment beforethe matches are considered in computing the corrective motion. Forexample, when a threshold of three measured points is required to matchto a segment, the matches between the measured points 1102G and 1102Hand the model points 1106I and 1106J, respectively, may be ignored. If,instead, the threshold value is one or two measured points, thesematches may he included in the computation. In various embodiments, thethreshold value may be 2, 3, 5, 10, or more points, and may depend onthe generation of the passageway modeled by the segment. For example, athreshold value associated with the segment 1104E may be lower or higherthan a threshold value associated with the segment 1104C because thesesegments are associated with different generations of passageways in theanatomic model 1104.

Referring now to FIG. 11D, shown therein are two associated anatomicmodels: model 1108 (the solid line model) and model 1110 (the dashedline model). The anatomic model 1108 is associated with a firstrespiratory phase, while the anatomic model 1110 is associated with asecond respiratory phase. For example, the first respiratory phase maybe an extreme of inhalation and the second respiratory phase is anextreme of exhalation while the patient P breathes. During naturallyoccurring processes in the body, organs, such as the lungs, may deform.In order to compensate for such processes, multiple models may begenerated as shown in FIG. 11D. In FIG. 11D, the set of measured points1102 includes points represented by data that contains a respiratoryphase marker. As depicted, the points 1102 include points illustrated inFIG. 11D as circles and points illustrated therein by crosses. Thepoints 1102 illustrated, as circles include a respiratory phase markerindicating inhalation. The points 1102 illustrated as crosses include arespiratory phase marker indicating exhalation. In some embodiments,there may be separate sets of measured points stored in memory.

When the points 1102 include respiratory phase markers, anotherheuristic may provide that the points be matched to the correspondingphase anatomic model. As shown in FIG. 11D, while the measured point1102I is closer to the segment, 111013, the medical system matches themeasured point 1102I to a point 1112A on the segment 1103E of theanatomic model 1108. This is because the measured point 11021 includes arespiratory phase marker for inhalation and the anatomic model 1108 is aphased model associated with the inhalation phase of respiration. Themeasured point 1102J is matched to a model point 1114A that is part ofthe segment 1110B of the anatomic model 1110 because the point 1102) isan exhalation point (e.g., the data representing point 1102) includes arespiratory phase marker indicating it was obtained during exhalation)and the anatomic model 1110 is a phased model associated with theexhalation phase. Similarly, because the measured point 1102L is anexhalation point, the point 1102I, matches to the model point 1114B. Thepoint 1102K snatches to the model point 1112B because they are bothassociated with the inhalation phase.

In embodiments where there are two phases represented in two models andthe measured points each have a binary phase marker, the process 712 ofFIG. 7 may be performed separately to register the first phase measuredpoints with the first phase model and to register the second phasemeasured points with the second phase model. Or the separate sets ofmeasured points may be registered with the corresponding model points.Where sets of measured points are stored separately in memory, the datarepresenting the points may not include a phase marker. In someembodiments, the phase marker of a given measured point may bedetermined by measuring the phase of a patient's respiration over timeand comparing this information with a timestamp associated with eachmeasured point. When the timestamp of a given measured point indicatesthat the injured point was collected at the peak of inhalation, the datarepresenting the measured point may be updated to include theappropriate phase marker. Additionally, in some embodiments in which thecatheter 610 includes a shape sensor, temporal shape data may be used todetermine the respiratory phase and include the appropriate markers andmeasured data points.

Referring now to FIGS. 11E and 11F, some embodiments of the medicalsystem may implement a heuristic of a maximum distance threshold suchthat measured points that match to model points but that have aseparation distance that is greater than the maximum distance thresholdare excluded from the computation of the corrective motion for aniteration. As shown in FIG. 11E, the measured points 1102. have been atleast initially registered with the anatomic model 1104 by a seedingprocess. One or more earlier iterations may have been performed as well.In some embodiments, a maximum distance threshold may he establishedthat is applied to the anatomic model 1104 generally. Alternatively, asillustrated, a different maximum distance threshold is associated witheach generation of the passageways in the anatomic model 1104. Themaximum distance threshold for each generation may be determinedaccording to characteristics such as passageway diameter and/ordeformability, which affect the likelihood that a point a given distancefrom the centerline is a legitimate point rather than an anomaly.

As illustrated in FIG. 11E, a first maximum distance threshold 1116A isassociated with the model segment 1104A, a second maximum distancethreshold 1116B is associated with the model segment 1104B, and a thirdmaximum distance threshold 1116C is associated with the model segment1104D. Because the model segment 1104C represents a passageway of thesame generation as the passageway represented by the model segment1104B, the second maximum distance threshold 1116B may also beassociated. with the segment 1104C. Similarly, the segments 1104E,1104F, and 1104G, are all associated with the same generation ofpassageway as the segment 1104D. Accordingly, the third maximum distancethreshold 1116C may also be associated with the segments 1104D, 1104F,and 1104G. These maximum distance thresholds 1116A, 1116B, and 1116C maycorrespond with an average passageway radius obtained from a pluralityof patients or they may be obtained from a model of the specific patientP.

In some embodiments, the maximum distance thresholds 1116A, 1116B, and1116C may be related to an average passageway radius obtained from thepatient P specifically. For example, the maximum distance thresholds maybe obtained based on passageway radius calculations from the segmentedmodel 502 illustrated in FIG. 5A. Because the more distal passageway'smay have smaller radii than the more proximal passageway's, the maximumdistance thresholds for the more distal passageways may; generally beless than the maximum distance thresholds for the more proximalpassageways.) In some embodiments, the. maximum distance threshold mayhe greater than approximately the diameter of the passageway greaterthan approximately twice the radius). In some embodiments the maximumdistance threshold may be approximately one and a half times the radiusof the passageway. In some embodiments, the maximum distance thresholdmay be about 20 mm for the trachea and as small. as around 2 mm fordistal passageway's. In some embodiments, the maximum distance thresholdmay decrease from the trachea for one or two generations and thenincrease again to account for the increased likelihood of deformation inthe more distal passageways.

As illustrated in FIG. 11E, the measured point 1102M is matched to apoint on the segment 1104A. However, the separation distance between themeasured point 1102M and the segment 1104A is greater than the maximumdistance threshold 1116A and so the match may be ignored when computingthe corrective motion to be applied to the set of measured points 1102.Similarly, the measured point 1102N is further away from the segment11040 than the maximum distance threshold 1116B and so may also beignored.

Referring now to FIG. 11F, shown therein is the result of the correctivemotion computed based on the set of measured points 1102 and theanatomic model 1104 as illustrated in FIG. 11E. Due to the correctivemotion, the measured points 1102M and 1102N are position within maximumdistance threshold 1116A and 1116B, respectively. Accordingly, thematches between the measured points 1102M and 1102N and the closestpoints in segments 1104A and 11040, respectively, may be included by'the maximum distance heuristic as factors in the computation of thecorrective motion of the subsequent iteration.

The maximum distance thresholds may be applied in other ways. Forexample, the maximum distance threshold may be provided as a function ofgeneration, or as a function of distance from a specific feature, suchas the main carina. The maximum distance threshold may also becalculated as a function of depth into the passageways. Accordingly, themaximum distance threshold may be larger at a proximal end of a modelsegment, such as the model segment 1116B, then at a distal end of thatmodel segment.

Referring now to FIG. 11G, in some embodiments if the point gatheringinstrument is not moved for a period of time, multiple points may bemeasured in a single location or very close together. As shown in FIG.11G, there is a cluster 1118 of measured points 1102. Because thecluster includes so many points, the collection of measured points atthe location of the cluster 1118 may skew the results of the computationto determine the corrective motion by including multiple matches, Inorder to filter out this result, the distance between the measuredpoints 1102 may be compared to identify clusters. Alternatively oradditionally, clusters, like the cluster 1118, may be identified bydetermining the distance between measured points that are in sequence bytimestamp. This may allow for the identification of a cluster caused bya pause in movement of the catheter 610 rather than by a narrowing of apassageway. As another heuristic, the measured points in the cluster1118 and/or associated matches may be ignored. In some embodiments, thecluster 1118 may be permanently deleted in order to avoid having toscreen out the cluster 1118 in subsequent iterations of the registrationprocess 712.

Referring now to FIG. 11H, shown therein in another heuristic that maybe used in matching measured points 1102 to the anatomic model 1104.FIG. 11H includes a plurality of artificial model points 1120 that aregenerated based on the anatomic model 11.04. The artificial model points1120 may be incorporated into the anatomic model 1104 for one or moreiterations before being discarded in subsequent iterations. Newartificial model points may be calculated for each iteration, in someembodiments. When the matching process is performed one or moreartificial points 1120 may be generated based on the measured points andthe radius of the matched passageway. For example, the artificial point1120A may be generated based on the measured point 1102P. As shown inFIG. 11H, the measured point 1102P is further away from its closestpoint on model segment 1104A (point 1106k) than the radius 11.22A. Theartificial point 1120A is created along the line between the model point1106 k and the measured point 1102P at a distance no greater than radius112A from the model point 1106 k. The artificial point 1102A may then beadded to the set of model points for the next iteration. For thatiteration the measured point 1102P is matched to the artificial point1120A and the resulting match is used to compute the corrective motionto be applied to the set of measured points 1102.

As shown in FIG. 11H, another artificial model point 1120B is generatedalong the line connecting the measured point 1102Q to the nearest modelpoint 1106 k. The artificial point 1120B is defined along that line at adistance, equal to radius 112B, from the model segment 1104C. As shownin FIG. 11H, the segments 1104A, 1104B, and 1104C include artificialpoints of the artificial model points 1120. The artificial model points1120 that border the segment 1104A are positioned up to a radialdistance 1122A away from the segment 1104A. While illustrated in twodimensions in FIG. 11H, the artificial model points 1120 may simulate atwo-dimensional plane or a volume. For example, the artificial point1120A may be closer to the model point 1106 k than the radius 1122A whenthe measured point 1102P is closer than the radius 1122A to the modelpoint 1106k. In such cases, the artificial point 1120A may be at thesame location as the measured point 1102P itself. Other heuristics maythen be applied to determine whether the matches between the measuredpoint 1102P and the artificial point 1120A and between the measuredpoint 11020 and the artificial point 1120B are included in thecomputation of the corrective motion. For example, if the distancebetween the artificial point 1120B and the measured point 1102Q isgreater than a maximum threshold distance, the match may be discardedfor that iteration.

In some embodiments, the heuristics above and other heuristics may beused in combination to prevent one or more measured points from beingmatched or to prevent one or more matches from being factored into thecomputation of the corrective motion. Thus, heuristics may be employedby the control system 112 in series (e.g., one heuristic per iteration)or in parallel (e.g., multiple heuristics operating in a singleiteration).

In some embodiments, other heuristics may be used to assigned weights tomeasured points. Additionally in some embodiments, the control system112 may guide a clinician in obtaining measured points to use inregistering and anatomic model. For example, certain passageways in theupper lobe of each lung may provide particularly reliable and usefulinformation for registering and anatomic model to a patient undergoing aprocedure. Accordingly, in some embodiments a user interface may bedisplayed by the control system 112 in the display system 110 to aclinician. The user interface may direct the clinician to steer acatheter into high information locations within the upper lobes of thelungs. In some embodiments, one or more measured points may be ignoredsuch that the measured points are not matched in a matching process tomodel points. In some embodiments, one or more matches may be ignoredsuch that the matches are not included in the computation of thecorrective motions. Thus, heuristics may be applied to points as well asto matches.

Referring again to FIG. 7, at process 724 the localized instrument maybe displayed with the anatomic model to assist the clinician in animage-guided surgery. FIG. 12 illustrates a display system 1200displaying, in a user interface, a rendering of anatomic passageways1202 of a human lung 1204 based upon anatomic model information. Withthe patient surgical space registered to the model space as describedabove in FIG. 10, the current shape of the catheter 610 and the locationof the distal end 618 may be located and displayed concurrently with therendering of the passageways 1202. The anatomic model information may heobtained from measured data points, moved data points, registered datapoints, and other data points described herein may be displayed on thedisplay 110 in a visual representation. The data points may be visuallyrepresented in a user interface by a plurality of points or dots on thedisplay or as a rendered model, such as a mesh or wire model createdbased on the set of data points. In some embodiments, a visualrepresentation may be refreshed in the display 110 after each processingoperations has been implemented to alter the data points.

As shown in FIG. 13, in some embodiment, the control system 112 mayadjust weights associated with one or more of the points to alter theireffects as factors in the computation of the corrective motions. Theweights may be adjusted based on one or more factors, Shown in FIG. 13is a set of measured points as stored in a point pool 1300 in memory ofthe control system 112. The exemplary point pool 1300 contains datarepresenting the measured points obtained from within the patient P. Inthe point pool 1300, each point includes a point identifier, a set ofcoordinates, a timestamp, a sensor ID, a phase marker, and a weight.Sonic embodiments may include the timestamp as the point identifierinstead of a separate point identifier. This data may be formatted inmany different ways. As shown in FIG. 13, the phase marker is a binaryvalue, either 0 or 1, depending on whether the phase is inhalation orexhalation, respectively. Other phase markers may be used in otherembodiments. The weights as shown in the exemplary point pool 1300 arenormalized weights. Some embodiments, such as those using ashape-sensing catheter may not include a sensor ID, which may be used bycatheters having a plurality of EM sensors or a plurality of otherdiscrete sensors that can be identified to determine approximately whereon the catheter the data point was measured.

In some embodiments of the method 700 of FIG. 7, the registrationprocess 712 may include an additional process in which weights formeasured points are altered according to parameters or rules. Theweights may be altered by adding a weight where no weight existed beforeor by changing the weigh associated with a given measured point. Forexample, in some embodiments more recently obtained points may beassigned a relatively higher weight. The relatively higher weight may beprovided by incrementally decreasing the weights assigned to lessrecently obtained points as more time passes. More recently measuredpoints may be more accurate due to movement of the patient, so byweighting the more recently measured points more than the less recentlymeasured points the registration process 712 may be biased to reflectthe most current information. The weights may be normalized weights ornon-normalized.

In some embodiments, the weight of a given measured point may be basedon the generation of the passageway in which the point was obtained. Forexample, as shown in FIG. 11A, the anatomic model 1104 includes severalsegments 1104A, 1104B, and 1104C and others that are associated withspecific generations of passageways. Because the trachea is a broaderpassageway, and so provides less information with which to register themeasured points to the modeled points, measured points that match to thesegment. 1104A he having a relatively reduced weight. Similarly,measured points that match to the segment 1104C and other of the samemore distal generation may be assigned a relatively reduced weight aswell. The measured points associated with the more distal generationsmay be assigned less weight because these more distal passageways aremore likely to deform due to the forces exerted by the point gatherinstrument on the passageways. The segment 1104B is associated with amore central passageway that is narrower than the segment 1104A and moreresilient to deformation by the catheter 610 than the segment 1104C.

In order to compensate for the deformation that can be caused by thepoint gathering instrument used to obtain the points in the point pool1300, the weight of each point may be based on the state of thecatheter. For example, the catheter may include a torque sensor, andwhen high torque is indicated a lower weight may he assigned as the hightorque may indicate significant deformation. Similarly, if the catheteris in a controlled state such that the distal end. of the catheter isactively positioned in a central portion of the passageway through whichis the catheter is passing the measured point may have a relativelyhigher weight than if the catheter is in a flaccid state. In someembodiments, the measured points may be collected into the point pool1300 only when the catheter is actively being steered and controlled inorder to collect points from the center of the passageways. When in theflaccid state the catheter may be more likely to pass along the bottomof the passageway than when in the active, controlled state. In someembodiments, the measured points collected in the flaccid or passivestate may be compensated with an adjustment to make it as if the pointswere collected closer to the center of the passageway. This may be doneby altering the coordinates of the collected points to move the pointtoward the center. Additionally, in order to minimize deformation causedby inserting a catheter further into passageways, some embodiments maylimit point collection to when the catheter is in a passive state and isbeing retracted from the passageways, When a camera is provided on thedistal end of the catheter, using image-recognition techniques thecontrol system may determine based on obtained images, whether thecatheter is in the middle of the passageway. The control system maylimit point collection to when the catheter is in the middle asindicated by image-recognition.

Weights may he assigned based on the, respiratory phase in which ameasured point is collected. Thus, when a single anatomic model is usedthe measured points that are similar in phase to the single anatomicmodel (like the anatomic model 1108 of FIG. 11D) may be given greaterweight than the measured points that are associated with a differentphase,

When measured points can be collected by the catheter at multiplelocations along its length, whether clue to the inclusion of ashape-sensor or a plurality of discrete sensing/transmitting devices,the history of the catheter may be used to assign weights. As thecatheter is advanced through the passageways of the patient's anatomy,multiple points are collected along its length. A less distal portion ofthe catheter may collect a point that has the same or substantiallysimilar coordinates as a point collected earlier by a more distalportion of the catheter. Alternatively, if the catheter is beingwithdrawn from a vessel rather than being advanced, the coordinates of apoint obtained by the distal end of the catheter may be the same orsubstantially similar to the coordinates of a point obtained earlier bya less distal portion of the catheter. In some embodiments, the measuredpoints are only collected using the catheter when the catheter is beingwithdrawn from the passageways of the patient. This history may indicatethat the recurring point is particularly reliable and the weights ofeither or both of the earlier obtained. point and the later obtainedpoint may be adjusted to be relatively higher. In some embodiments, thepoints measured by more distal portions of the catheter may be weightedhigher than less distal portions of the catheter, because the lessdistal portions of the catheter may be thicker in diameter and morelikely to cause deformation of tissue. Other configurations of thecatheter may also be used as factors upon which to base weights formeasured points.

In some embodiments. machine learning may be used to identify qualitiesof the most reliable points. The control system may then apply weightsaccordingly. Each of the described factors may be used to determine theweight of a single point, Thus, while a single factor may he used todetermine the weight of a given point in some embodiments, in otherembodiments multiple factors may be used by the control system to adjustthe weight of one or more of the points in the point pool 1300.

Referring again to FIGS. 6A and 6B, shown therein is a tracking device624 temporarily affixed to the patient P. By monitoring the trackingdevice 624 the control system may determine whether the patient P moves.The tracking device 624 may be a device capable of generating positionand or movement data, such as a set of EM sensors/transmitters,accelerometers, etc. In some embodiments the tracking device 624 may hea passive device that is monitored by a monitoring device. For example,the tracking device 624 may be a pad that is easy to identify by avisual tracking system. For example, the pad may have known dimensionsand may include a distinctive pattern and/or color to allow machinevision to monitor its location. When the tracking device 624 moves, thevisual tracking system may provide an indication to the control systemthat the patient P has moved.

Due to movement of the patient P, a previous registration between ananatomic model and measured points may become less accurate. Accordinglyinformation obtained from and/or displayed in connection with theanatomic model, such as a lesion or tumor, may not be accuratelycommunication to a clinician. In some embodiments, after a satisfactoryregistration has been obtained and movement of the patient P isdetected, the registration process may begin again. In some embodiments,a change in displacement and/or orientation measured by the trackingdevice 624 may be used to update the registration. In some embodiments,the registration process 712 may be performed again beginning with aseeding process. In other embodiments, the registration process 712 maybe performed without performing a new seeding process. For example, ifthe movement of the patient P is determined to be small, discardingolder measured points from the point pool 1300 (or decreasing theirrelative weighting substantially in favor of points obtained after thedetection of the movement of patient P) and collecting new measuredpoints using the catheter. Thus in some embodiments, a new set of pointsis collected and used to register the model to the moved patient P or amixed weighting of new and old points may be used. In some embodiments,a notification is provided to a user to initiate a registration due topatient movement. In other embodiments, registration may be initiated bythe control system 112 after motion in the tracking device 624 isdetected.

In some embodiments, the shape of the catheter after the movement of thepatient P may be used to compensate for errors in the rigid registrationdue to the motion. In some embodiments, the shape sensor data may beused to provide new measured points after the movement of the patient Pthat may be used to perform further registration processes, like thosein the method 700.

In some embodiments, the point gathering instrument is coupled to ateleoperational robotic arm. The teleoperational arm may move accordingto commands from an operator input system. In some embodiments, theinsertion stage 608 may be mounted on a teleoperational arm. When thearm or the insertion stage 608 moves, the movement may he communicatedto a control system from encoders incorporated into the setup joints ofthe arm and/or the stage 608. When the insertion stage 608 and/or theteleoperational arm move as indicated by the encoders, the registrationprocess may he performed again. A comparison between the commandedmotion and. the measured motion (with the shape sensor) may indicatethat movement has occurred that was not commanded. If the comparisonvalue between the commanded motion and the measured motion exceeds athreshold value, all or a portion of the registration process may bereinitiated. This may occur, for example, if the distal tip wascommanded to enter the opening of a passageway but is dislodged from theentrance to the opening due to anatomical motion, tissue texture, orother anatomic forces.

Referring now to FIG. 14, a flowchart of a method 700A is shown therein.The method 700A shares many of the processes included in the method 700as described herein and as illustrated in FIG. 7. The shared processesinclude the same reference numerals. The method 700A further includes aset of processes that provide a method for initiating the collection ofmeasured points within the patient passageway. The method 700A may beimplemented as part of a workflow managed by the control system 112 toenable clinicians to more effectively and efficiently treat patientslike the patient P.

The collection of location and/or shape data to generate a set ofmeasured points describing the passageways of the patient P's anatomymay begin automatically when the point gathering instrument (e.g., thecatheter 610) is introduced into the patient passageways. This may beperformed at process 1402 in which the control system 112 detects one ormore point collection conditions that trigger the collection of measuredpoints. For example, when the catheter 610 includes a camera or imagesensor, images may be collected and processed to determine when thedistal end 618 enters the patient passageways. For example, the controlsystem 112 or another component of the system 100 may identify theappearance of the main carina obtained by the image sensor system or mayidentify the trachea from the colors of images obtained therein.

Also, as described herein in connection with FIGS. 6C and 6D, the ETtube 622 may include colors and or patterns on the interior surface 623thereof. When images obtained from the catheter 610 during introductioninto the patient passageways do not include the colors and/or patternsof the interior surface 623, this may provide a point collectioncondition. When information obtained from the images indicates that thetransition from the interior of the ET tube 622 to the interior to thetrachea (or other anatomic passageway) is completed, the pointcollection condition. may be detected.

Additionally, as described in connection with FIGS. 6A and 6B, theposition measuring device 620 provides information about the position ofthe rigid instrument body 612 as it moves on the insertion stage 608along an insertion axis A. The movement of the rigid instrument body 612may provide an indication of the movement of the distal end 618 of thecatheter 610. This kinematic information may be used to infer that,after a certain amount of insertion motion, the catheter 610 is likelyto have entered patient passageways. Thus, the kinematic information mayprovide a collection condition that, when detected, can trigger theprocess 1404 of initiating the collection of location information tocollect a set of measured points describing the patient passageways.Similarly, where the catheter 610 is coupled to another robotic devicesuch as a robotic arm, the kinematic information provided by encoders inthe arm of device may be used to detect a point collection condition,such as an amount of movement in a specific direction. In someembodiments, the movement commands provided to the robotic device or armmay also be used to identify a point collection condition that triggersthe collection of points at process 1404, rather than relying onencoders to relay the actual movements. In some embodiments, both thecommanded motion and the measured motion of the robotic device may beused. When sufficient motion is detected and/or commanded, the controlsystem 122 may detect the motion and determine whether it satisfiesrequirements of a point collection condition.

Although the systems and methods of this disclosure have been describedfor use in the connected bronchial passageways of the lung. they arealso suited for navigation and treatment of other tissues, via naturalor surgically created connected passageways, in any of a variety ofanatomic systems including the colon, the intestines, the kidneys, thebrain, the heart, the circulatory system, or the like.

At the process 1404, the data representing the points may be added tothe point pool 1300, stored in memory, as shown in FIG. 13. Thislocation information represented by the measured points may he collectedexclusively at the distal end 618 of the catheter 610, from a pluralityof discrete points along the length of the catheter, or obtained fromshape-sensor information continuously along the length of the catheter610. When the shape-sensor information provides position informationcontinuously along the length of the catheter 610, the shape-sensorinformation may he sampled to generate discrete points, in sonic,embodiments. Depending on how the points are obtained from the catheterand the type of catheter used for point collection) the detection of apoint collection condition for collection of points from a certainportion of the catheter may not be sufficient to trigger the collectionof points at another portion of the catheter, For example, when thedistal end 618 transitions from the ET tube 622. to the trachea, thecondition may be satisfied so as to trigger collection of points at thedistal end 618, but not at a portion of the catheter that is still inthe ET tube 622. Thus, in some embodiments, after the distal end 618exits the tube as recognized by the visual information provided to thecontrol system 112, kinematic information may he used to determine whento begin point collection from less distal portions of the catheter.When the catheter 610 includes a plurality of EM sensors distributedalong its length, the distance of insertion may be used to determinewhen to begin collecting information. For example, if the second EMsensor is positioned 2cm from a first EM sensor at the distal end of thecatheter, the point collection condition for the second EM sensor may bethat the first EM sensor must have exited the ET tube 622 and thecatheter must have been moved 2 cm or another predetermined distance.The movement is obtained from kinematic information as described herein.

In some embodiments, the registration process, like that described inprocess 712 of FIG. 7, may be repeated during the course of a surgicalprocedure. As described, a movement of the patient P may trigger a newregistration process. In some embodiments, the registration algorithm isexecuted by the control system 112 periodically, such as every minute orevery five minutes during the procedure. In other embodiments, theregistration process may be constantly run on the control system 112 asa background process and continuously updated. For example, even after aregistration process is complete, the collection of measured points inthe point pool 1300 may continue as long as a catheter capable ofobtaining the measured points (from any suitable modality) is presentwithin the patient passageways. In some embodiments in which theregistration process operates as a background process, the registrationmay only be updated when a new registration is superior to an oldregistration as determined by an accuracy metric. The accuracy metricmay he the percentage of measured points successfully matched to themodel or may be the average distance between the measured points and thematched model points. Other accuracy metrics may be implemented in otherembodiments. Thus, when a more recent registration has a greaterpercentage of measured points successfully matched, the more recentregistration may he used instead of an older registration. By requiringthat an accuracy metric be met in order to replace one registration withanother, consistency is maintained unless a superior registration isobtained.

In some embodiments, when a later registration replaces an earlierregistration or an earlier registration is deemed replaceable by thecontrol system 112 with a later registration, an alert may he providedto the clinician through a user interface to indicate that there is achange in registration or that there is a superior registrationavailable. In some embodiments, the control system 112 may requireclinician approval through the user interface before the superiorregistration is implemented. For example, when a superior registrationis identified an alert may be rendered to the display system 110 alongwith a button or other user interface element by which the clinician canapprove or disapprove to the new registration. The new registration willthen be implemented or not depending on the clinician's decision.

One or more elements in embodiments of the invention may be implementedin software to execute on a processor of a computer system such ascontrol system 112. When implemented in software, the elements of theembodiments of the invention are essentially the code segments toperform the necessary tasks. The program or code segments can he storedin a non-transitory processor readable storage medium or device,including any medium that can store information including an opticalmedium, semiconductor medium, and magnetic medium. Processor readablestorage device examples include an electronic circuit; a semiconductordevice, a semiconductor memory device, a read only memory (ROW, a flashmemory, an erasable programmable read only memory (EPROM); a floppydiskette, a CD-ROM, an optical disk, a hard disk, or other storagedevice. The code segments may be downloaded via computer networks suchas the Internet, Intranet, etc. As described herein, operations ofaccessing, detecting, initiating, registered, displaying, receiving,generating, determining, moving data points, segmenting, matching, etc.may he performed at least in part by the control system 112 or theprocessor thereof.

Note that the processes and displays presented may not inherently herelated to any particular computer or other apparatus. The requiredstructure for a variety of these systems will appear as elements in theclaims. In addition, the embodiments of the invention are not describedwith reference to any particular programming language. It will beappreciated that a variety of programming languages may be used toimplement the teachings of the invention as described herein.

While certain exemplary embodiments of the invention have been describedand shown in the accompanying drawings, it is to be understood that suchembodiments are merely illustrative of and not restrictive on the broadinvention, and that the embodiments of the invention not be limited tothe specific constructions and arrangements shown and described sincevarious other modifications may occur to those ordinarily skilled in theart.

1-46. (canceled)
 47. A system comprising: a catheter; a display system;and a control system in communication with the catheter and the displaysystem, the control system including one or more processors configuredfor: receiving a set of model points of a model of one or morepassageways of a patient; receiving a first set of measured pointscollected from within the patient passageways, each point comprisingcoordinates within a surgical environment occupied by the patient;generating a first registration between the set of measured points andthe set of model points; generating a second registration between asecond set of measured points and the set of model points; anddetermining whether to implement the second registration in place of thefirst registration.
 48. The system of claim 47, wherein the second setof measured point includes at least one of the first set of measuredpoints and additional measured points collected from within the patientpassageways.
 49. The system of claim 47, wherein determining whether toimplement the second registration in place of the first registrationcomprises comparing an accuracy metric of the first registration with anaccuracy metric of the second registration.
 50. The system of claim 49,wherein the accuracy metric is a percentage of measured points matchedto the model points.
 51. The system of claim 49, wherein the accuracymetric comprises at least one of: an average distance between themeasured points and the model points and a distance from a seed using inperforming an initial registration.
 52. The system of claim 47, whereinthe control system is further configured for: determining an accuracymetric of the first registration and determining an accuracy metric ofthe second registration.
 53. The system of claim 47, wherein thegenerating the second registration between the second set of measuredpoints and the set of model points comprises collecting additionalmeasured points for a predetermined period of time or collecting apredetermined number of additional measured points.
 54. The system ofclaim 47, wherein the control system is further configured for:requesting, through a user interface, authorization from a clinicianprior to implement the second registration.
 55. The system of claim 47,wherein the control system is further configured for: providing, througha user interface, an alert to a clinician that the second registrationis available for implementation in place of the first registration. 56.The system of claim 47, wherein the control system is further configuredfor: detecting a motion of the patient.
 57. The system of claim 56,wherein detecting the motion of the patient comprises detecting arelative motion between the patient and a proximal end of the catheter.58. The system of claim 57, wherein the catheter comprises a shapesensor extending a length of the catheter.
 59. The system of claim 56,wherein the second set of measured points comprises points measuredwithin the patient passageways after the detection of the motion of thepatient.
 60. The system of claim 56, wherein a marker temporarilyaffixed to the patient is tracked to monitor for motion of the patient.61. The system of claim 56, wherein the second registration is used tocompensate for errors in the first registration that are attributable tothe motion of the patient.
 62. The system of claim 56, wherein themotion is detected in shape-sensor data obtained by a shape sensor inthe catheter while the catheter is positioned within the passageways ofthe patient.
 63. A system comprising: a catheter; a display system; anda control system in communication with the catheter and the displaysystem, the control system including one or more processors configuredfor: receiving a set of model points of a model of one or morepassageways of a patient; determining a state of a catheter positionedwithin the one or more passageways of the patient; when the state of thecatheter satisfies a condition, collecting a set of measured points fromwithin the patient passageways, each point comprising coordinates withina surgical environment occupied by the patient; and generating aregistration between the set of measured points and the set of modelpoints.
 64. The system of claim 63, wherein determining the state of thecatheter comprises determining whether the catheter is in a passivestate or an actively controlled state, and wherein the conditionrequires a passive state for collection of a point of the set ofmeasured points.
 65. The system of claim 63, wherein determining thestate of the catheter comprises determining whether the catheter isbeing inserted into the patient passageways or being withdrawn from thepatient passageways, and wherein the condition requires that thecatheter is being withdrawn from the patient passageways for collectionof a point of the set of measured points.
 66. The system of claim 63,wherein determining the state of a catheter comprises determiningwhether a distal end of the catheter is positioned within a center ofone of the patient passageways, and wherein the condition requires thatthe catheter be in the center of one of the patient passageways forcollection of a point of the set of measured points.